MATLAB: Saving many variables as NetCDF makes file too large. How to reduce

large file sizenetcdfsaving files

Hello all,
I would REALLY appreciate any help on this, as I looked online but didn't find straight answers. I am trying to save many variables of different sizes as NetCDF format in this way:
numrow = 576;
numcol = 48;
numlayers = 125;
numtime_save = numtime + 1;     %This is usually a value of '8'
%%Define the dimensions
netcdf.setDefaultFormat('NC_FORMAT_CLASSIC') ; 
ncid = netcdf.create(files_name,'NC_WRITE');
dimidrow = netcdf.defDim(ncid,'rows',numrow);
dimidcol = netcdf.defDim(ncid,'length',numcol);
dimidlay = netcdf.defDim(ncid,'layers',numlayers);
dimidtime = netcdf.defDim(ncid,'time',numtime_save);
%%Define the name of variables, with dimensions
%%Question: do these all really have to be double precision???
varmask = netcdf.defVar(ncid,'cloudmask','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_sigma = netcdf.defVar(ncid,'cloudmask_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_clear = netcdf.defVar(ncid,'cloudmask_clear','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_clear_sigma = netcdf.defVar(ncid,'cloudmask_clear_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_radar = netcdf.defVar(ncid,'cloudmask_radar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_radar_sigma = netcdf.defVar(ncid,'cloudmask_radar_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_both = netcdf.defVar(ncid,'cloudmask_both','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_both_sigma = netcdf.defVar(ncid,'cloudmask_both_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_lidar = netcdf.defVar(ncid,'cloudmask_lidar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varmask_lidar_sigma = netcdf.defVar(ncid,'cloudmask_lidar_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varheight = netcdf.defVar(ncid,'height','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varheight_sigma = netcdf.defVar(ncid,'height_sigma','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
varN = netcdf.defVar(ncid,'N','NC_BYTE',[dimidrow dimidcol dimidtime]);
%%end the variable naming and dimensions, then actually input
%%variables into netcdf file, then close the netcdf file
netcdf.endDef(ncid);
netcdf.putVar(ncid,varmask,cloudmask);
netcdf.putVar(ncid,varmask_sigma,cloudmask_sigma);
netcdf.putVar(ncid,varmask_clear,cloudmask_clear);
netcdf.putVar(ncid,varmask_clear_sigma,cloudmask_clear_sigma);
netcdf.putVar(ncid,varmask_radar,cloudmask_radar);
netcdf.putVar(ncid,varmask_radar_sigma,cloudmask_radar_sigma);
netcdf.putVar(ncid,varmask_both,cloudmask_both);
netcdf.putVar(ncid,varmask_both_sigma,cloudmask_both_sigma);
netcdf.putVar(ncid,varmask_lidar,cloudmask_lidar);
netcdf.putVar(ncid,varmask_lidar_sigma,cloudmask_lidar_sigma);
netcdf.putVar(ncid,varheight,height);
netcdf.putVar(ncid,varheight_sigma,height_sigma);
netcdf.putVar(ncid,varN,N);
netcdf.close(ncid);
clear mex
However, when I save it this way, the file size is 1.33 GB! That is huge…If I save the files as:
save(filename, 'cloudmask','cloudmask_sigma','cloudmask_clear','cloudmask_clear_sigma',...
    'cloudmask_radar','cloudmask_radar_sigma','cloudmask_both','cloudmask_both_sigma',...
    'cloudmask_lidar','cloudmask_lidar_sigma','height','height_sigma','N');
this file has a size of 16 MB. I would REALLY like to save in NetCDF format, because my boss needs it this way. I have many files to save, this is just an example but it is inside of a loop. PLEASE HELP!

Best Answer

%Example for one variable. Have to do on all
numrow = 576;
numcol = 48;
numlayers = 125;
numtime_save = numtime + 1; %This is usually a value of '8'
comp = 5; %compression size, for example
%%Define the dimensions
netcdf.setDefaultFormat('NC_FORMAT_CLASSIC') ;
ncid = netcdf.create(files_name,'NC_WRITE');
dimidrow = netcdf.defDim(ncid,'rows',numrow);
dimidcol = netcdf.defDim(ncid,'length',numcol);
dimidlay = netcdf.defDim(ncid,'layers',numlayers);
dimidtime = netcdf.defDim(ncid,'time',numtime_save);
varmask_radar = netcdf.defVar(ncid,'cloudmask_radar','NC_FLOAT',[dimidrow dimidcol dimidlay dimidtime]);
netcdf.defVarDeflate(ncid,varmask_radar,true,true,comp);
netcdf.endDef(ncid);
netcdf.putVar(ncid,varmask_radar_sigma,cloudmask_radar_sigma);
netcdf.close(ncid);
clear mex
Related Question