I don't think there is any evidence of dimming over thousands or millions of years. Initially, the dimming were relatively short events, over several days and they included massive dips in the star's light, up to 22%. A planet passing in front of a star dims it by only a fraction of a percent, so this is much, much larger than a planet. At the time, the theory was a large comet field that orbits the star and periodically blocks out 22% of the light for a few days.
Now, new evidence shows that not only are these short dimming events occurring, but a long, slow, and steady dimming has been occurring over the past 100 years. This dimming has reduced the star's light by 20% over that time. This type of dimming would be almost impossible to be comets, since the field would need to be unimaginably large to cause that much consistent dimming over such a long period of time.
Occam's Razor suggests both the long-term dimming and the short-term dimming are caused by the same thing, so if the long-term dimming is almost impossible to be comets, the short-term dimming is likely not comets either.