Python LASPy – How to Update EVLR Header Info for .LAS File Using LASPy

Question

I'm trying to change the header info for a LAS file using the python 2.7 (64-bit) laspy module (version 1.2.5). I'm attempting to follow the example shown here (in the section "Writing Data + EVLRS"): http://laspy.readthedocs.io/en/latest/tut_part_3.html

However, when I update the Extended Variable Length Record (EVLR) info with the new content, I get this error message:

Error text: ValueError: offset must be non-negative and no greater than >buffer length (1094)

This error shows up on line 151 of laspy/base.py:

    self._pmap = np.frombuffer(self._mmap, self.pointfmt,
                offset = self.manager.header.data_offset,
                count = self.manager.header.point_records_count)

The "buffer length" value of 1094 seems to be the same as the length of the EVLR text record, while the "data_offset" value above is a much higher value of 4151.

import numpy as np
import laspy
import shutil
import datetime

print('Update started: {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))

 # Set input / output filenames
evlrContentFile = "./LAS_DATA/evlrContentFile.txt"
inFileName = './LAS_DATA/infile.las'
outFileName = './LAS_DATA/outfile.las'

# Get EVLR string content
with open(evlrContentFile, "r") as evlrFile:
    evlrString=evlrFile.read();

# make a copy
shutil.copy(inFileName,outFileName);

# open the outfile for modification
outFile_v14 = laspy.file.File(outFileName, mode = "rw")

# create a new EVLR header entry
new_evlr = laspy.header.EVLR(user_id = 10, record_id = 2,
                VLR_body = evlrString)

# update the outfile EVLR header entries
old_evlrs = outFile_v14.header.evlrs
old_evlrs.append(new_evlr)
outFile_v14.header.evlrs = old_evlrs
outFile_v14.close()
print('Update Completed: {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))

I want to modify the existing files with the new header info with a minimum of processing, so I want to avoid using laspy to copy the contents of the LAS points to the new output file if possible. Hence why I am using the "rw" modify option instead of "w" to create the output file.

Answer 1

OK, I think I've solved my problem.

The issue is noted at the github page for laspy here: https://github.com/grantbrown/laspy/issues/51

I needed to adjust the "start_first_evlr" property to be the full length of the LAS file as shown below. Not sure if that is the best way, but it seems to work. I needed to read the mmap size for the input LAS file (before the EVLR was appended) to get the index where the EVLR should be placed.

# update the "start_first_evlr" property if this is the first EVLR.  Otherwise the laspy code will start with an offset of 0...
if (len(inFile_v14.header.evlrs) == 0):
    theReader = laspy.base.Reader(inFileName,"r")
    theDataProvider = laspy.base.DataProvider(inFileName, theReader)
    theDataProvider.open("r")
    theDataProvider.map()
    endofthefile = theDataProvider._mmap.size()
    outFile_v14.header.start_first_evlr = endofthefile
    theDataProvider.close()
    theReader.close()

I don't think I should have had to do this, though... The laspy code in base.py seems to want to reset the mmap buffer to the size of the first EVLR (disregarding the point data content when the new EVLR is added). This takes place around line 904 (snippet shown below) in base.py in the "set_evlrs" method. The "dat_part_1" object should contain all the LAS point info up to the end of the input file. However the "old_offset" value is set to 0 without the code I added to my script above (where I set the "start_first_evlr" property).

        self.data_provider.fileref.seek(0, 0)
        dat_part_1 = self.data_provider.fileref.read(old_offset)
        # Manually Close:
        self.data_provider.close(flush=False)
        self.data_provider.open("w+b")
        self.data_provider.fileref.write(dat_part_1)
        total_evlrs = sum([len(x) for x in value])
        self.data_provider.fileref.write("\x00"*total_evlrs)
        self.data_provider.fileref.close()
        self.data_provider.open("r+b")
        self.data_provider.map()
        self.seek(old_offset, rel = False)

        for evlr in value:
            self.data_provider._mmap.write(evlr.to_byte_string())

        if self.has_point_records:
            self.data_provider.point_map()
        self.populate_evlrs()

The full attached code seems to work:

 # Set input / output filenames
evlrContentFile = "./LAS_DATA/evlrContentFile.txt"
inFileName = './LAS_DATA/infile.las'
outFileName = './LAS_DATA/outfile.las'

print('Update started: {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))

# Get EVLR string content
with open(evlrContentFile, "r") as evlrFile:
    evlrString=evlrFile.read();

# open input LAS file
inFile_v14 = laspy.file.File(inFileName, mode = "r")

# make a copy of the input file
shutil.copy(inFileName,outFileName);

# modify 'rw' the copied file
outFile_v14 = laspy.file.File(outFileName, mode = "rw")

# create new EVLR record
new_evlr = laspy.header.EVLR(user_id = EVLR_userid, record_id = EVLR_recordid,
                VLR_body = evlrString)

# update the "start_first_evlr" property if this is the first EVLR.  Otherwise the laspy code will start with an offset of 0...
if (len(inFile_v14.header.evlrs) == 0):
    theReader = laspy.base.Reader(inFileName,"r")
    theDataProvider = laspy.base.DataProvider(inFileName, theReader)
    theDataProvider.open("r")
    theDataProvider.map()
    endofthefile = theDataProvider._mmap.size()
    outFile_v14.header.start_first_evlr = endofthefile
    theDataProvider.close()
    theReader.close()

# outFile_14 has the same, empty EVLR as inFile
old_evlrs = inFile_v14.header.evlrs
old_evlrs.append(new_evlr)

# update the EVLR property
outFile_v14.header.evlrs = old_evlrs

# close and exit
outFile_v14.close()
inFile_v14.close()
print('Update Completed: {:%Y-%m-%d %H:%M:%S}'.format(datetime.datetime.now()))