I consider myself very lucky to be able to train martial arts with my family, I highly recommend it. John Roseberry-Shihan likes to say "The family that kicks together, sticks together." We were the first family in the 38 year history of Sho-Rei-Shobu-Kan to receive our black belts in karate at the same time, and I hope there will be many more in the future. My daughter also trains jiu-jitsu with me. I believe a great part of the closeness we have as a family, and our ability to communicate well with one another, can be attributed to training together. Besides, there's nothing better than being mad at your husband and taking it out on him when you spar. Also, if you train with your children, as long as you are in class it's not child abuse. Be warned however, because as they get older (and so do you), it quickly turns into parental abuse instead.
The only drawback is that you may find your lives taken over by the dojo. For instance, before we started karate, the Ethridge's used to go camping sometimes. I miss camping. It's worth it though, because not only has the dojo brought me closer to my own family, it has given me a whole new family with which to grow.