ZFS performance: slow writes and file unlinks compared to UFS

Hi,

We need help tuning ZFS, because our application is experiencing much slower writes and file unlinks on ZFS compared to UFS.  The writes are asynchronous.

ZFS is not allowing us to keep up with the rate of incoming data.  We started using ZFS on 5/17/2017.  We are currently ingesting data

on both UFS and ZFS in order to keep up.  We believe we need to migrate to ZFS (more detail in the "why switch" section below).

This is a long post, so I've broken it up into the following sections:

•     System Configuration
•     Application Overview
•     Preliminary Analysis
•     Stats Collection
•     DTrace Issues