I agree, but it made me think. Maybe it's a bit pedantic, but wouldn't it be more accurate to say people are born as a blank slate concerning basically everything. But is the initial absence of even thinking about "where do humans and the universe come from" necessarily the same as being atheist? Not thinking about all these questions, and not being told/taught what to think, I guess means not believing in a god. But until a person actually starts asking themselves these questions, can you really say they're atheist?
Of course eventually in modern (secular) schools they will eventually be taught about evolution, but hypothetically a child could come to all kinds of conclusions or belief systems if left on their own.
I'd personally shy away from calling a baby an atheist. I get the distinction between implicit and explicit that someone else mentioned in reply to you but just don't see it as useful seeing you could also say the same thing about a huge number of concepts. As someone who lacks belief in deities myself it kind of smacks as people trying to claim others for our 'club' too, to me at least.
All dogs are atheists too, at least I assume they are if we're going down that road but I don't see the usefulness of stating that.
Relevant to this topic, no baby believes in LGBT rights. We acknowledge that they aren't able to conceptualise what they are and don't put a label on them.
I think my beef is with the word atheism really. There are no equivalent words for someone who doesn't believe in LGBT rights on the basis that they are unable to conceptualise what they are due to being too young, through severe mental impairment, or growing up in an extremely isolated community with little to no contact with the outside word. For those who are capable we'd likely call them homophobic, but there's no a-LGBT-rights-ism word or whatever it would be.
Of course there is a difference too from believing in the existence of something or not and believing that something is a good, just, or worthy concept or not. Important to point that out.
Atheist/ism really is a unique word as far as I can think of. There isn't another commonly used one that people use to denote someone who doesn't believe in the existence of something or what the actual state of reality is that many/most do believe believe in that readily springs to mind. We don't call flat earthers a-round-earthers or even anti-round-earthers to give them the equivalent of anti-theists. No word for round earth agnostics either for the few (if they exist) who'd say we cannot know. Maybe not the greatest example seeing as flat earthers are making a positive claim. I don't know, a-santa-claus-ists?
Edit: Sorry for random Saturday morning ramblings!