Hello:
I need to process a 62 GB structured binary file written from a 24 h simulation. The structure of the file is as follows:
ft(1).length = 1; ft(1).type = 'integer*4'; ft(1).name = 'VehID'; ft(2).length = 1; ft(2).type = 'real*4'; ft(2).name = 'Time'; ft(3).length = 1; ft(3).type = 'integer*4'; ft(3).name = 'Longitude'; ft(4).length = 1; ft(4).type = 'integer*4'; ft(4).name = 'Latitude'; ft(5).length = 1; ft(5).type = 'integer*2'; ft(5).name = 'Heading'; ft(6).length = 1; ft(6).type = 'integer*4'; ft(6).name = 'Segment'; ft(7).length = 1; ft(7).type = 'integer*2'; ft(7).name = 'Dir'; ft(8).length = 1; ft(8).type = 'integer*4'; ft(8).name = 'Lane'; ft(9).length = 1; ft(9).type = 'real*4'; ft(9).name = 'Offset'; ft(10).length = 1;ft(10).type = 'real*4'; ft(10).name = 'Distance'; ft(11).length = 1;ft(11).type = 'real*4'; ft(11).name = 'Speed'; ft(12).length = 1;ft(12).type = 'real*4'; ft(12).name = 'Acceleration';
I am able to read this file using readfields with the format above but it is taking forever to go through its 1,506,979,651 records. I would like to partition this file in 96 files based on the value of 'Time', which covers 24 hours (15 min increments -> 96 files), and keep only VehID, Time, Distance, Speed, and Acceleration. After extensive readings (I am still learning Matlab), I understand memmapfile would be a good way to go, but I am unable to make that command work. I would need help to write the appropriate memmapfile statement (especially the format) so I can process this file efficiently. Thank you for your help,
JDS
Best Answer