Perplexed on logging both text and binary information to a file

15 Dec 2008 10:20 PM

Andrew Falanga

Hi,

On the heals of my previous post I'm now asking for the proper way of
writing text and binary data to a log file. I have a class, in the
singleton pattern, that will be used for logging program operations to
a file. I'm having trouble with using the two.

I eventually ended up at a point of using a FileStream object as a
"base" and then, in my class, also having a BinaryWriter and a
StreamWriter object both taking the FileStream object as an argument
for construction. I ended up with some very odd results when trying
to use these two objects in tandem.

I then noticed that the BinaryWriter object had member functions that
took string objects and such and decided to use only a BinaryWriter
object. This has proven unproductive too. The string objects I pass
are being written to the log file as text but there is extra
characters in the mix. What am I missing? Is this something with the
encoding of the file (UTF8 and so forth)?

I expect that I will be primarily logging text to this log file, but I
can see that the need can arise, as I write this application, that
writing binary data will be necessary. I need to write it into the
same file. Any help is most appreciated.

Thanks,
Andy

15 Dec 2008 10:30 PM

Jeff Johnson

Show quote Hide quote

"Andrew Falanga" <af300***@gmail.com> wrote in message
news:d59a2343-a99b-4742-a1b5-67c7d7737f20@y1g2000pra.googlegroups.com...

> On the heals of my previous post I'm now asking for the proper way of
> writing text and binary data to a log file. I have a class, in the
> singleton pattern, that will be used for logging program operations to
> a file. I'm having trouble with using the two.
>
> I eventually ended up at a point of using a FileStream object as a
> "base" and then, in my class, also having a BinaryWriter and a
> StreamWriter object both taking the FileStream object as an argument
> for construction. I ended up with some very odd results when trying
> to use these two objects in tandem.
>
> I then noticed that the BinaryWriter object had member functions that
> took string objects and such and decided to use only a BinaryWriter
> object. This has proven unproductive too. The string objects I pass
> are being written to the log file as text but there is extra
> characters in the mix. What am I missing? Is this something with the
> encoding of the file (UTF8 and so forth)?
>
> I expect that I will be primarily logging text to this log file, but I
> can see that the need can arise, as I write this application, that
> writing binary data will be necessary. I need to write it into the
> same file. Any help is most appreciated.

Who is this log file for? Logs are traditionally text-based so that they can
be read by humans. If you need to log some non-text data, you often do so
with a "hex dump."

If you DO want to log binary data but also have human-readable text in
there, I recommend that you use an Encoding class. Encoding.Default is the
best choice in my opinion because it will return the default ANSI encoding
for your system locale. Then you can run your string through the GetBytes()
method and get back an array of bytes which, when written to your log,
should show up without any extra "fluff" when viewed in a text editor.

On the other hand, if you want to write everything as plain text, let me
know, because I have a routine which creates hex dumps and I'll post it for
you.

16 Dec 2008 12:02 AM

Andrew Falanga

Show quote Hide quote

On Dec 15, 3:30 pm, "Jeff Johnson" <i....@enough.spam> wrote:
> "Andrew Falanga" <af300***@gmail.com> wrote in message
>
> news:d59a2343-a99b-4742-a1b5-67c7d7737f20@y1g2000pra.googlegroups.com...
>
>
>
> > On the heals of my previous post I'm now asking for the proper way of
> > writing text and binary data to a log file. I have a class, in the
> > singleton pattern, that will be used for logging program operations to
> > a file. I'm having trouble with using the two.
>
> > I eventually ended up at a point of using a FileStream object as a
> > "base" and then, in my class, also having a BinaryWriter and a
> > StreamWriter object both taking the FileStream object as an argument
> > for construction. I ended up with some very odd results when trying
> > to use these two objects in tandem.
>
> > I then noticed that the BinaryWriter object had member functions that
> > took string objects and such and decided to use only a BinaryWriter
> > object. This has proven unproductive too. The string objects I pass
> > are being written to the log file as text but there is extra
> > characters in the mix. What am I missing? Is this something with the
> > encoding of the file (UTF8 and so forth)?
>
> > I expect that I will be primarily logging text to this log file, but I
> > can see that the need can arise, as I write this application, that
> > writing binary data will be necessary. I need to write it into the
> > same file. Any help is most appreciated.
>
> Who is this log file for? Logs are traditionally text-based so that they can
> be read by humans. If you need to log some non-text data, you often do so
> with a "hex dump."

For people to read. Though I expect to log nearly always text, for
the purpose this program must fill, I might have cases where I'll have
to log some of the data that I'm reading from the input sources
(serial ports and the like). This data won't always contain text and
I can see the need to log non-text data for analysis of issues. At
this point, I'm more concerned with having the functionality in place
so as to be able to do it when I get into coding the "actual" code of
the program.

>
> If you DO want to log binary data but also have human-readable text in
> there, I recommend that you use an Encoding class. Encoding.Default is the
> best choice in my opinion because it will return the default ANSI encoding
> for your system locale. Then you can run your string through the GetBytes()
> method and get back an array of bytes which, when written to your log,
> should show up without any extra "fluff" when viewed in a text editor.

One of my attempts, before posting, was doing something like this and
then passing that byte array to a StreamWriter object (I think). I'm
not sure of the web page I was on with that code example and I'm very
much a novice at all this (my background is entirely UNIX and C++).
Can you provide an example of how this would be done?

>
> On the other hand, if you want to write everything as plain text, let me
> know, because I have a routine which creates hex dumps and I'll post it for
> you.

If the byte arrays, encoding objects and GetBytes() routines aren't
adequate or the right way to do this, that may be helpful. Thanks for
the offer.

Andy

16 Dec 2008 2:33 PM

Jeff Johnson

Show quote Hide quote

"Andrew Falanga" <af300***@gmail.com> wrote in message
news:1a3e0f4b-d965-409b-9788-4183972ffa26@i24g2000prf.googlegroups.com...

>> If you DO want to log binary data but also have human-readable text in
>> there, I recommend that you use an Encoding class. Encoding.Default is
>> the
>> best choice in my opinion because it will return the default ANSI
>> encoding
>> for your system locale. Then you can run your string through the
>> GetBytes()
>> method and get back an array of bytes which, when written to your log,
>> should show up without any extra "fluff" when viewed in a text editor.

> One of my attempts, before posting, was doing something like this and
> then passing that byte array to a StreamWriter object (I think). I'm
> not sure of the web page I was on with that code example and I'm very
> much a novice at all this (my background is entirely UNIX and C++).
> Can you provide an example of how this would be done?

How about this: can you post the code you're currently using to write
strings? Then we can dissect it.

16 Dec 2008 9:50 PM

Andrew Falanga

Show quote Hide quote

On Dec 16, 7:33 am, "Jeff Johnson" <i....@enough.spam> wrote:
> "Andrew Falanga" <af300***@gmail.com> wrote in message
>
> news:1a3e0f4b-d965-409b-9788-4183972ffa26@i24g2000prf.googlegroups.com...
>
> >> If you DO want to log binary data but also have human-readable text in
> >> there, I recommend that you use an Encoding class. Encoding.Default is
> >> the
> >> best choice in my opinion because it will return the default ANSI
> >> encoding
> >> for your system locale. Then you can run your string through the
> >> GetBytes()
> >> method and get back an array of bytes which, when written to your log,
> >> should show up without any extra "fluff" when viewed in a text editor.
> > One of my attempts, before posting, was doing something like this and
> > then passing that byte array to a StreamWriter object (I think). I'm
> > not sure of the web page I was on with that code example and I'm very
> > much a novice at all this (my background is entirely UNIX and C++).
> > Can you provide an example of how this would be done?
>
> How about this: can you post the code you're currently using to write
> strings? Then we can dissect it.

Sorry I did not reply to this sooner. The place I work has some
rather restrictive rules regarding source code. So, I can't post.
However, I've managed to solve the problem. I've been able to
adequately accomplish what I needed by using a FileStream object and
using the following (actually taken from MSDN) to accomplish the
majority of what I need:

// assume this is part of a properly defined class
// with a FileStream member object

public void Log(String s) {
    // using the classes FileStream member fs
    byte[] tmp = new UTF8Encoding(true).GetBytes(s);
    fs.Write(tmp, 0, fs.Length);
    fs.Flush();
}

// and then, for writing the binary
public void WriteBytes(byte[] ba) {
    fs.Write(ba, 0, fs.Length);
    fs.Flush();
}

// and then, for writing the bytes of a byte array as text
public void WriteBytesAsText(byte[] ba) {
    String tempStr = "";
    for(int i = 0; i < ba.Length; i++)
        tempStr += ba[i].ToString();

    byte[] tempBA = new UTF8Encoding(true).GetBytes(tempStr);
    fs.Write(tempBA, 0, tempBA.Length);
    fs.Flush();
}

Basically, I use a similar process as above to convert the byte array
into hex characters using System.Convert.ToString(byte val, int base)
when I'd like to dump the contents as hex text.

Thanks for the help here and in the other thread I posted. It was
these threads that helped me solve this problem in part. I would
welcome any suggestions on how to better do this.

Andy

15 Dec 2008 11:00 PM

Phil Wilson

Something that sometimes works if you have binary data that is wrapped in an
object is to write your own ToString() method to turn the content into a
human readable string.

--
Phil Wilson
Definitive Guide to Windows Installer
http://www.apress.com/book/view/1590592972

Show quoteHide quote

"Andrew Falanga" <af300***@gmail.com> wrote in message
news:d59a2343-a99b-4742-a1b5-67c7d7737f20@y1g2000pra.googlegroups.com...
> Hi,
>
> On the heals of my previous post I'm now asking for the proper way of
> writing text and binary data to a log file. I have a class, in the
> singleton pattern, that will be used for logging program operations to
> a file. I'm having trouble with using the two.
>
> I eventually ended up at a point of using a FileStream object as a
> "base" and then, in my class, also having a BinaryWriter and a
> StreamWriter object both taking the FileStream object as an argument
> for construction. I ended up with some very odd results when trying
> to use these two objects in tandem.
>
> I then noticed that the BinaryWriter object had member functions that
> took string objects and such and decided to use only a BinaryWriter
> object. This has proven unproductive too. The string objects I pass
> are being written to the log file as text but there is extra
> characters in the mix. What am I missing? Is this something with the
> encoding of the file (UTF8 and so forth)?
>
> I expect that I will be primarily logging text to this log file, but I
> can see that the need can arise, as I write this application, that
> writing binary data will be necessary. I need to write it into the
> same file. Any help is most appreciated.
>
> Thanks,
> Andy

15 Dec 2008 11:11 PM

Arne_Vajhøj

Andrew Falanga wrote:
Show quoteHide quote

> On the heals of my previous post I'm now asking for the proper way of
> writing text and binary data to a log file. I have a class, in the
> singleton pattern, that will be used for logging program operations to
> a file. I'm having trouble with using the two.
>
> I eventually ended up at a point of using a FileStream object as a
> "base" and then, in my class, also having a BinaryWriter and a
> StreamWriter object both taking the FileStream object as an argument
> for construction. I ended up with some very odd results when trying
> to use these two objects in tandem.
>
> I then noticed that the BinaryWriter object had member functions that
> took string objects and such and decided to use only a BinaryWriter
> object. This has proven unproductive too. The string objects I pass
> are being written to the log file as text but there is extra
> characters in the mix. What am I missing? Is this something with the
> encoding of the file (UTF8 and so forth)?
>
> I expect that I will be primarily logging text to this log file, but I
> can see that the need can arise, as I write this application, that
> writing binary data will be necessary. I need to write it into the
> same file. Any help is most appreciated.

Always log text. If you have binary data you need to log then
convert them to hex string first.

Arne

16 Dec 2008 3:12 AM

Tim Jarvis

Arne Vajhøj wrote:

> Always log text. If you have binary data you need to log then
> convert them to hex string first.
>
> Arne

I totally agree with Arne on this, you might also try Base64 for your
binary data.

16 Dec 2008 3:48 AM

Arne_Vajhøj

Tim Jarvis wrote:

> Arne Vajhøj wrote:
>> Always log text. If you have binary data you need to log then
>> convert them to hex string first.
>
> I totally agree with Arne on this, you might also try Base64 for your
> binary data.

If the log is to be read by a program, then Base64 will save 1/3
compared to hex.

But if it is to be read by humans, then hex is 1000 times easier
to read than Base64.

Arne