Well, it depends on what your definition of elite is. Duke's definitely struggled in the tournament since their last Final Four appearance in 2004, but not that many teams have been consistent over that stretch. The four years since have sent 12 teams to the Final Four; only three have made it more than once and seven haven't even made the tournament in some of the other years. If Duke's no longer elite, who is? UNC, Kansas, and UCLA are pretty obvious. Memphis, probably, although the conference schedule is a little sad. Florida has two national titles, but hasn't seen the tournament since, and now may lose their coach. Connecticut has the championship for the year of Duke's last Final Four appearance, but since has a postseason performance almost identical to the Blue Devils (Four tournament appearances and one absence, which counting last night's performances, netted two Elite Eight trips.) Who else are you counting as elite, though? Keep in mind the guys from Durham have averaged only seven losses a season these past four years (and almost 28 wins). If Duke doesn't make the cut, it's a precious little club.
(Even if you back it out to Elite Eight teams over the last four years, that's 22 teams with only seven repeat offenders. This ain't beanbag, folks.)
If anything, the recent Duke woes remind me of the stretch for Dean Smith from 1985 to 1990, where the Heels had five top ten finishes in six years but never advanced past the Elite Eight and managed only one ACC Championship. There was a fair amount of talk that Smith was over the hill and should retire, and yes, that the team was no longer "Elite." UNC persevered and spent the rest of Smith's career making it to four Final Fours in seven years and cutting down the nets on New Orleans. As much as I'd rather it not be the case, I don't think this is the last we've seen of Duke competing at a national level.