Yes you can install Endeca Server in an existing WebLogic 10.3.6, although it is preferable to dedicate a WebLogic domain to hosting only the Endeca Server application. See this topic about the Endeca Server and WebLogic integration, from the Endeca Server 7.6.0 doc.
In it, it is stated:
"Important: For performance and troubleshooting purposes, it is preferable to dedicate a WebLogic domain to hosting only the Endeca Server application. In this case, the endeca-server root context is used in the URL pointing to any of the WebLogic Server instances hosting the Endeca Server application."
Additionally, the Endeca Server Installation Guide provides instructions for installing the generic version of the Oracle WebLogic 10.3.6. installer package, and not any other WebLogic packages. Specifically, it states:
"This procedure relies on those components that are available in the WebLogic Server installation under the Oracle Restricted Use License. If your site already has access to the fuller version of the WebLogic Server, additional components may be available to you. However, the instructions in this topic do not guarantee that any other components of the WebLogic Server can be used with the Endeca Server. Additionally, certain components, such as the Enterprise Manager, are not discussed in this guide."