Everything is a compromise in video formats.
The original DVD formats give the best quality but gobble up gigabytes of storage, so they aren't that good for storing of lots of DVD's.
H264 format is probably about the most compact common format while maintaining reasonable quality video, and AVS can handle this.
However, if you use it in full HD resolution (1920 X 1080) it takes up hours of processor time to convert a full dvd, specially if it's not the latest super fast computer.
If you cut back the resolution a bit, say to 720, then it speeds up considerably.
Whatever you convert it to, make sure that the media player can decode it because if it can't decode H264 then it will be a wasted exercise.
Avi format is pretty safe in this respect so maybe you should stick to that.
Although I think H264 will eventually become more widely used because the file size is about half as big for the same quality.

Whatever you recode, keep the original DVD files (VOB, IFO BUP etc,) off the DVDs stored as data files on separate discs so when new formats come out you can recode them to whatever you want.
Blank DVD's are dirt cheap so it's no big deal to have a box of them stored somewhere.