First off, I agree with Dave - on the 'go for it' side of things, and the idea that it's all about supervision. I think a lot of stuff with kids is also all about moderation - too much TV, too much computer / gaming / iPod Touching, too many sweets or too much junk food etc. - all bad if we let them run amok - but I think we all do our best to not let that occur.
Also, I have a 6 year old daughter who recently got my V1 iPhone as a hand-me-down when I got the 3GS - with its phone service turned off so it is now a glorified Touch. She loves it and is a heck of a knowledgeable little user already. She enjoys games, books, email back and forth to her grandparents and others, and googling for things she wants to learn about (or just to see pictures of horses).
I think if you keep an eye on things the Touch can be a great source of fun and education for your kids. Oh, and I agree that your kids richly deserve a special present right about now.