if you have a compatible character set and don't use features that don't exist in XE, there shouldl be no problem. Depending on your table definition and stored data it might be necessary to redefine VARCHAR2 columns. Most people don't define their VARCHAR2 columns simply with a "number" as length which defaults to "byte" instead of "characters". So if you have special characters that would use only one byte in your source charset but might need two in UTF-8, you might run into problems in the import. The solution would be to resize your VARCHAR2 columns that can contain special characters to store the appropriate amount of characters instead of bytes.
Can a DMP file be taken from a 10gR2(SE1) export and imported into 11g Express?
In general export and import can be used between releases. E.g. from Upgrade guide:
In a case when going from a higher edition, there may be features missing in the target edition.
Did you attempt a migration yet?