ORA-01841: (full) year must be between -4713 and +9999, and not be 0

Need help on date comparison as getting ORA-1841 in 19c(19.14.0.0.0)

Working Query:

   SELECT       partition_name,      TO_CHAR(to_date(substr(partition_name,instr(partition_name, '_',1,2)+1,6)  ,'YYMMDD'),'DD-MON-YYYY')  as_of,       TO_NUMBER(TO_CHAR(to_date(substr(partition_name,instr(partition_name, '_',1,2)+1,6)  ,'YYMMDD'),'J')) as_of_num

          FROM       dba_segments

    WHERE      segment_name NOT LIKE 'BIN%'       and partition_name LIKE 'P2%_2106%'

      and segment_type='TABLE SUBPARTITION'       and rownum < 5;

               -------------

 P2_109294_210630_D 30-JUN-2021 2459396

P2_109294_210630_D 30-JUN-2021 2459396

P2_109294_210630_D 30-JUN-2021 2459396

P2_104202_210630_D 30-JUN-2021 2459396

When i add same in to the WHERE cluase endup having error:

    SELECT       partition_name,