Building something that works for everyone is a noble thing to do. It's one of the strengths of the web, even though many people keep forgetting how important it is to cater to a potential audience of billions. Accessibility is essential even when your clients don't believe it is, that's one of the pledges any front-ender should make. But sometimes accessibility fanatics lose sight of the bigger picture.
Accessibility is not about building websites for minorities. It's about building a website that is the perfect compromise, so that quality levels for all types of visitors are as high as possible. This means making choices that may not be beneficial for some parties, but help out a great deal with another portion of your audience. Percentages play a big part in these decisions (you can't cripple 95% of your audience to give a little extra benefit to the other 5%).
Why am I ranting about this? Well, assistive technology (like screen readers) isn't always top notch and up to date with the latest standards. Some html structures may be easier for people working with a screen reader, but might compromise semantic validity, maybe even compromising grammatical rules. At moments like these choices have to be made and sometimes that means that people using a screen reader will have a slightly harder time navigating your website. In return you can comply with the rules and be sure that nothing will go wrong for future generations of browsers while crawlers can still effectively interpret your html code.
Accessibility is there for everyone, from your average lowlife html crawler to the cutting edge web developer using the latest version of Chrome.