If it weren't Madeline, it would probably have been some other kid.
If they wanted to abduct a child, they would have.
Irresponsible, maybe. But primarily responsible, no way.
And if you're telling me that Madeline could open them shutters, and just disappear of the face of the world, without anybody noticing her walking about alone... dillusional.