after long reading about class loaders and complex situation where two objects might have the same name and such, I wonder if would it be right to assume that I should have a class loader for each defined task, for example importing plugins, importing classes from server, and others?
Since "task" can be defined as wide and as narrow as you wish, that doesn't really mean a lot.
You rarely need to define any classloaders at all. Just use what the environment provides.
The exception here are systems such as modular plugin systems (OSGi for example), where you'll do lots of classloader magic.