Your mistake was changing the first line from the original "default vesamenu.32" to "default memtest". This line should not have been changed.
If you want the default menu option to be memtest, you have two choices:
1) You can reorder the set of label entries to make memtest the first label entry item which appears in the list. (You probably don't want to do this as it will make the menu appear rather odd looking.) Isolinux normally considers the first label entry in the list as default.
2) You can add a line within the label entry for memtest that says "menu default" i.e. something like this:
menu label memtest - Run memtest
I believe that adding "menu default" will make memtest the selected entry when the menu appears, and it will be booted after the timeout period has expired unless the user makes a change by moving the selection bar.