Transitioning NetSuite’s database from the old version to NS2 is far from straightforward. As I discussed in my previous post, numerous variables come into play, including changes in field and table names and structural differences. Additionally, differing values within fields and table linkages necessary to retrieve data that was inherently present in the original tables under the old DB structure significantly elevate the migration risk.
Our team is still in the midst of this data migration process, and I'd like to share some essential tips for others undertaking similar migrations:
1. Rethink the Migration Approach Initially, I questioned whether to adapt the migration to fit the old table structures to enable a quicker transition to post-DWH outputs like BI reports or interfaces. However, I soon concluded that it’s more effective to start fresh by establishing new tables in the NS2 structure. This approach involves mapping out and prioritizing the critical fields we currently utilize for reports and interfaces, focusing on them in the initial phase.
2. Handling Historical Data Deciding whether to migrate historical data or link to previous tables for creating views with UNION ALL was a significant consideration. We discovered discrepancies in some field values, making it impractical to maintain the old structure alongside new values. Therefore, it’s advisable to migrate all historical data comprehensively. Our evaluations indicated no loss of historical data—only previously deleted records were removed from the old tables and do not exist in the new structure. NetSuite’s focus appears to be on more efficient DB maintenance rather than supporting DWH and BI processes.
3. Data Extraction Strategy: Delta vs. Full Choosing between delta (incremental) and full data extractions depends on table size. Generally, for large tables, delta extraction is preferable, while small tables can be handled with full reads. However, for specific tables like transactionline, we encountered an unbridgeable gap. Records deleted via the UI are removed from the table itself, which delta reads cannot capture due to the lack of a deleted status indicator. To address this, we opted to perform full reads covering the past six months and replace records based on keys, ensuring no deleted records remain as active entries. This approach eliminates the need to maintain historical deletions.
4. Establishing Bridging Views After setting up the new tables, we plan to create views that bridge the reports layer and existing interfaces, preventing the need for additional migrations.
Important Tips for a Successful Migration:
- Map Critical Fields and Tables: Understand their business significance and the values they represent, Try to focus on used Tables and Fields in your Organization and migrate them on Step one and try to include keys fields for future addons areas as possible
- Maintain an Internal Catalog: Keep a comprehensive catalog of tables and fields within NetSuite, including any changes and tables specifically tailored to your organization.
https://5330119.app.netsuite.com/app/recordscatalog/rcbrowser.nl?whence=
- Utilize Helpful Tools: Consider using extensions like the Chrome extension “Netsuite Field Explorer” that can extract and breakdown NetSuite UI fields into table field names for better understanding.
https://chromewebstore.google.com/detail/netsuite-field-explorer/cekalaapeajnlhphgdpmngmollojdfnd?hl=en&pli=1
- Engage a NetSuite Consultant: Having an expert can aid in managing permissions and identifying tables, as new naming conventions may use internal numbering rather than business-friendly names.
- Use NetSuite "Oracle Community" Guru and tips
https://community.oracle.com/netsuite/english/post/discussion/community-tips
Migrating to NS2 presents substantial challenges, but with careful planning and the right strategies, it can lead to a more efficient and robust data management system.
Feel free to reach out if you’re navigating a similar migration or have any questions!
Eyal Rosenfeld
Director Analytics and Business Intelligent
Zadara
