[langsec-discuss] delimited base64 file specification

lee hughes toxicnaan at gmail.com
Wed Mar 20 09:02:28 UTC 2013


Having thought more about it, i agree :-) !

Size should be left to some 'other layer', be it file-system,  network , or
something else!.

However its not unusual for structured data to have length, or even
checksum/hash/digial signature
so, the receiver can check the integrity of the data. But you can argue
that some other layer should take care of this too. but does it?

Take zfs, this uses sophisticated methods to prevent bit rot, however if i
were to copy that 'file' to say FAT32, then we loose this mechanism. If
these
features are part of the file format, then the file can keep the same
'features', no matter where is resides, (memory, disk, network,
cloud, parallel dimension).

However if base64 is contains a zip or other archive format, then the
underlying zip would carry some of this information. :-)

I'm with keep it simple. The more 'features' you come up with, the more
complex things become, and, the more it's left to miss interpretation.

Perhaps if all early end systems and serial links could of supported full
handle 8 bit bytes and their associated readers could handle full binary
streams, we wouldn't have base64  formatting at all :-). I presume all
base64 was devised for was to handle binary streams, keeping them
compatible (visible) in 7bit ascii mail readers, so we could just 'attach'
a base64 stream inline with the message. :-). I presume 7bit ascii code was
chosen, because some serial links were 7 bit, with parity and a stop bit.
(my memory is a bit hazy on this).

anyway, nice work! :-)

Cheers,




On Wed, Mar 20, 2013 at 5:58 AM, Michael Ossmann <mike at ossmann.com> wrote:

> On Tue, Mar 19, 2013 at 02:33:38PM +0000, lee hughes wrote:
> >
> > Is there any limit on record size? i.e from null, to zero, to
> > 1000000Gigabytes.
>
> no
>
> > most protocol's/datafile formats (rightly or wrongly), give the
> > receiver a clue about sizes of the message.
>
> Many protocols do, but most file formats I can think of (certainly csv)
> don't.  My thinking is that keeping track of file size is the job of
> whatever framing is done around the file (if it needs to be done at
> all).  When stored on a filesystem, that is the filesystem's job.  If
> encapsulated in a protocol, that is the encapsulating layer's job (e.g.
> Content-Length headers).
>
> I toyed with the idea of having a table delimiter to permit multiple
> tables within a single file (with a new header record and a different
> number of fields permitted for each table) but decided that was an
> unnecessary complication since we can assume that there is some
> filesystem or other framing to separate things.
>
> With this format I'm trying to "do one thing and do it well" where that
> one thing is safe and simple storage of structured data, specifically as
> an interchange format where csv files are traditionally used.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.langsec.org/pipermail/langsec-discuss/attachments/20130320/fdb7a99d/attachment-0001.html>


More information about the langsec-discuss mailing list