Basically, illegal drugs are just the drugs that the wealthy do not use?

No, illegal drugs are drugs that the medical community have found dangerous and without valid use, like crack cocaine. The viewpoint you express is cynical and sarcastic, and would form a self-excusing base from which to break the law and use illegal drugs.