PeopleTools and Lifecycle Management - PSFT (MOSC)

MOSC Banner

PeopleSoft Search Framework - Elastic Indexing rate is slowing down as it progresses

I am trying to index 114mil rows of data and 41.5 attachments with 150mil rows of psfile_attdet data from a PeopleSoft CRM application to ES 6.1 (due to compatibility with PeopleSoft, can't upgrade to higher versions).

Elastic Configuration:

A single node (Master/Coordinator, ingest and data node) cluster with 200 primary shards with no replica
OS: Oracle Linux 5.4.17-2136.336.5.1.el7uek.x86_64 x86_64
OCPU: 32
Memory: 128GB
Disk: 10TB

elasticsearch.yml options:
http.max_content_length: 300mb
indices.memory.index_buffer_size: 20%
bootstrap.memory_lock: true

jvm.options

-Xms31g

-Xmx31g

-XX:ParallelGCThreads=20

-XX:NewRatio=2

refresh_interval : -1 » changed to 120s

Number of primary shards: 200

Howdy, Stranger!

Log In

To view full details, sign in to My Oracle Support Community.

Register

Don't have a My Oracle Support Community account? Click here to get started.

Category Leaderboard

Top contributors this month

New to My Oracle Support Community? Visit our Welcome Center

MOSC Help Center