[langsec-discuss] delimited base64 file specification

Michael Ossmann mike at ossmann.com
Wed Mar 20 05:58:36 UTC 2013


On Tue, Mar 19, 2013 at 02:33:38PM +0000, lee hughes wrote:
>
> Is there any limit on record size? i.e from null, to zero, to
> 1000000Gigabytes.

no

> most protocol's/datafile formats (rightly or wrongly), give the
> receiver a clue about sizes of the message.

Many protocols do, but most file formats I can think of (certainly csv)
don't.  My thinking is that keeping track of file size is the job of
whatever framing is done around the file (if it needs to be done at
all).  When stored on a filesystem, that is the filesystem's job.  If
encapsulated in a protocol, that is the encapsulating layer's job (e.g.
Content-Length headers).

I toyed with the idea of having a table delimiter to permit multiple
tables within a single file (with a new header record and a different
number of fields permitted for each table) but decided that was an
unnecessary complication since we can assume that there is some
filesystem or other framing to separate things.

With this format I'm trying to "do one thing and do it well" where that
one thing is safe and simple storage of structured data, specifically as
an interchange format where csv files are traditionally used.


More information about the langsec-discuss mailing list