ora-27100: shared memory realm already exists after setting sga_max_size

 Am running Oracle 10.2.0.4 on Windows server 2003 and wanted to increase the sga_max_size and put the db into archivelog mode. Ran the command for the sga_setting then tried to shutdown the db prior to running alter database archivelog.

SQL> shutdown immediate;
Database closed.
Database dismounted.
ORA-00600: internal error code, arguments: [LibraryCacheNotEmptyOnClose]

could not get any response so tried restarting the server. But oracle did not restart. If I try Startup I get ORA-27100: shared memory realm already exists. I tried renaming the spfile so the db would start up with the old setting in the pfile but always get the same error.