Watch "Baseball"
Now.
They're not immune to getting things wrong. Some particularly controversial documentaries of his were on the Civil War, WWII, and Jazz. However, the production values are absolutely insane. They actually make you feel like you're in that time period going through the events of the documentary. They also cover some very important issues and make you think on them.
If you're looking for the nitty gritties of a particular event, look somewhere else, but there is hardly a better primer for any topic than a Burns documentary.
They're very informative, detailed and the musical choices really set unique a unique mood for each one. You should start off with his Civil War documentary. The imagery, music and written statements make for a compelling watch. His prohibition documentary also explored a lot of the forces behind prohibition and how it basically injected steroids into gang crime in certain respects.
Really gripping stuff
The best, bar none.