For a couple years I've dealt with a problem in some of my 3D tools that need to parse configuration strings to extract data and rules. The functions I've used generally are the MemStream interfaces in 3ds Max's MAXScript.

Unfortunately, there is one glaring problem with the MemStream, and that is with seemingly arbitrary limit on the number of characters that can be processed with the function that is most useful in MemStream: MemStream.ReadBlock(). This method is hard-coded to fail with any block of data that returns more than 5120 characters. My best guess is that it was a result of memory bottlenecks in the computers of 2001 when this code was last updated.

The methods I've found online inevitably lead to Unknown System Exception errors.

Until Autodesk decides to update this, or someone wants to recompile the functions to be more robust, here is the workaround that I've found to be the fastest and reliable solution.

function wallworm_readBlock &memstr p1:"[" p2:"]" = (
  local startPos = memstr.pos()
  outputstring = memstr.readblock p1 p2
  if outputstring == undefined then (
   outputstring = ""
   local endPos = memstr.pos()
      memstr.seek startPos #seek_set
   local limit = memstr.size()
   if endPos > limit then (
    endPos = limit
   )
   while memstr.pos() < endPos do (
    append outputstring (memstr.readLine()+"\r\n")
   ) 
  ) else (
   outputstring = replace_LF_with_CRLF outputstring
  )
  outputstring 
   )


This function has solved almost all of my problems with using MemStream objects that have data blocks larger than 5120 characters.

Usage

To use this code, simply replace any code that calls Readblock() as such:

Original Code

 textStream = memStreamMgr.openString variableContainingString
 while textStream.eos() == false do (
  local blockData = textStream.readBlock "[" "]"
 )

New Code

 textStream = memStreamMgr.openString variableContainingString
 while textStream.eos() == false do (
  local blockData = wallworm_readBlock &textStream
 )