Earlier this week, I quickly read through Peter Vogel’s most recent piece in the Visual Studio Magazine. The article’s title is “Debugging Tools for the .NET Developer”. The article is very well written and had a lot of pointers and tools references.
One of the tools mentioned in Lizard Lab’s LogParser Lizard.
The article is available @ http://visualstudiomagazine.com/Articles/2011/11/01/Debugging-Tools-for-the-NET-Developer.aspx?Page=1
and the paragraph that appeared readily actionable is:
When it comes to reading those log files, I use Log Parser Lizard from Lizard Labs. The free version has some locked features that are only available if you buy the product, which is about $25, but I haven’t felt the need yet. Log Parser Lizard uses a SQL-like syntax to query log files (including CSV and XML files) and understands the IIS, Windows event logs and the log4net file formats straight out of the box. The results are displayed in a grid….
Sometime about database query via SQL and grid as opposed to loading log files into your favorite editor (Notepad++ in my case) and attempting to find data via regular text search is technically progressive.
So once in front of a computer, downloaded Lizard Lab’s LogParser and played with reading data from my local box event’s viewer. Everything went effortless easy.
As I use Log4Net quite a bit played with the built-in sample, as well.
But, as life is always GOOD and BAD, had problems when I tried accessing one of the log files that I generated with log4Net.
There is a log4Net’s demo bundled with the Application and the App read it quite gently. But, the ones I generated was a different story. Wish I could have conformed the format of my log file to match the author’s, but the App did not document the format of the sample log file.
I thus searched the .Net for common log file patterns. BTW, these patterns are known as “conversionPatterns” in the log4Net parlance.
These days it seems everyone has a pattern:
Apache.org have sample ones @ http://logging.apache.org/log4net/release/manual/configuration.html
<layout type="log4net.Layout.PatternLayout"> <conversionPattern value="%date [%thread] %-5level %logger [%ndc] - %message%newline" /> </layout>
SAS has sample ones – Saw one @ http://support.sas.com/documentation/cdl/en/logug/61514/HTML/default/viewer.htm#a003261250.htm
<layout> <param name="ConversionPattern" value="%d %-5p %t %c (%F:%L) %m" /> </layout>
The more I read about patterns the more it seems I will have flexibility in terms of what metadata I will want to place in my configuration specification and resultant log files.
And, only those will better appreciation for obscurity than I will love Regular Expression. Only a 4th night ago I found out to my total disamay that Microsoft.Net Libraries does not have C-Style sscanf as a built in function.
And, thus started my irresistible push and pull to XML log files. At least, they will have more\better rigidity in terms of what goes in a log file. And, be a bit self described.
<appender name="FileAppender" type="log4net.Appender.FileAppender"> <file value="C:\Temp\app-log4net.xml" /> <appendToFile value="false" /> <layout type="log4net.Layout.XmlLayoutSchemaLog4j"> <locationInfo value="true" /> </layout> </appender>
Tried the syntax pasted below to load\access my XML file via Lizard Lab’ Log Parser:
SELECT * FROM 'C:\Temp\app-log4net.xml'
But, no luck
Error parsing query : Error loading document <filename> : Reference to undeclared namespace prefix: ‘log4j’, [Unknown Error]
<a href=”https://danieladeniji.files.wordpress.com/2011/11/referencetoundeclarednamespaceprefix-log4j.jpg”><img class=”alignright size-full wp-image-1970″ title=”Reference To Undeclared Namespace Prefix – log4j” src=”https://danieladeniji.files.wordpress.com/2011/11/referencetoundeclarednamespaceprefix-log4j.jpg” alt=”” width=”640″ height=”202″ /></a>
Went back and reviewed the XML file and found the offending “log4j” tags:
<log4j:event logger=”dbEntail” timestamp=”1322168530203″ level=”INFO” thread=”1″><log4j:message>In objEntail.get:- SQL is dbo.usp_Entail</log4j:message><log4j:properties><log4j:data name=”log4net:UserName” value=”labuser” /><log4j:data name=”log4jmachinename” value=”labmachine” /><log4j:data name=”log4japp” value=”labQuery.exe” /><log4j:data name=”log4net:HostName” value=”labmachine” /></log4j:properties><log4j:locationInfo class=”dbSupport” method=”get” file=”c:\Support.cs” line=”200″ /></log4j:event>
<log4j:event logger=”dbEntail” timestamp=”1322168530203″ level=”INFO” thread=”1″><log4j:message>In objEntail.get:- SQL is dbo.usp_purchase</log4j:message><log4j:properties><log4j:data name=”log4net:UserName” value=”labuser” /><log4j:data name=”log4jmachinename” value=”labmachine” /><log4j:data name=”log4japp” value=”labQuery.exe” /><log4j:data name=”log4net:HostName” value=”labmachine” /></log4j:properties><log4j:locationInfo method=”get” file=”c:\Support.cs” line=”278″ /></log4j:event>
And, so next in line was to determine whether I can add the corresponding namespace references for log4j:
Found it difficulty to offhandedly add them to generated log files (thoroughly read through log4net docs) and also to Lizard’s Lab logparser (from per-using the environment configuration tools and no progress while trying to augment the SQL Query).
Probably there are many avenues to “forcefully” remove log4j from our logfile, I settled on trying on use search\replace type commands. Or, in this case search and discard.
- Tried Dos Commands ( http://www.dostips.com/?t=Batch.FindAndReplace )
- Tried simple text replacement utilities – Fart-it ( http://sourceforge.net/projects/fart-it/support )
- Tried Farting the easy way ( http://emtunc.org/blog/03/2011/farting-the-easy-way-find-and-replace-text/) . Worked well with text files, but could not get it to effectively help with my XML file
- Powershell – Got word that PowerShell works well, but unlikely to use it; the reason being that Powershell codes deployment are sometimes encumbered via MS Windows Group Policy code signing restrictive policy
- Found out that TexFinderX works http://sw.ixoft.com/texfinderx and http://www.addictivetips.com/windows-tips/find-and-replace-text-inside-multiple-files-using-texfinderx, but seems to be GUI only and do prefer command line interface for quicker & tidier access
- Tried VBScript code published by ScriptingGuy ( http://blogs.technet.com/b/heyscriptingguy/archive/2005/02/08/how-can-i-find-and-replace-text-in-a-text-file.aspx )
So the Text replacement code we finally settled on is:
How Can I Find and Replace Text in a Text File?
OPTION EXPLICIT Const ForReading = 1 Const ForWriting = 2 Set objFSO = CreateObject("Scripting.FileSystemObject") Set objFile = objFSO.OpenTextFile("C:\Scripts\Text.txt", ForReading) strText = objFile.ReadAll objFile.Close strNewText = Replace(strText, "Jim ", "James ") Set objFile = objFSO.OpenTextFile("C:\Scripts\Text.txt", ForWriting) objFile.WriteLine strNewText objFile.Close
Customized and added a bit of flexibility:
- Replaced input and output file name with command line arguments
<pre> Const ForReading = 1 Const ForWriting = 2 Const strText = "log4j:" Const strTextReplacement = "" Dim strFileInput Dim strFileOuput Dim objFSO Dim objFile Dim strBuffer Dim strBufferNew Dim strLog If (WScript.Arguments.Count <> 2) Then Wscript.Echo "Error: Invalid number of arguments entered. " strLog = "Usage: " + WScript.ScriptName + " InputFileName OutputFileName" Wscript.Echo strLog Wscript.Quit Else strFileInput = WScript.Arguments.Item(0) strFileOuput = WScript.Arguments.Item(1) End if Set objFSO = CreateObject("Scripting.FileSystemObject") Set objFile = objFSO.OpenTextFile(strFileInput, ForReading) strBuffer = objFile.ReadAll objFile.Close Set objFile = Nothing strBufferNew = Replace(strBuffer, strText, strTextReplacement) If objFSO.FileExists(strFileOuput) Then Set objFile = objFSO.OpenTextFile(strFileOuput, ForWriting) Else Set objFile = objFSO.CreateTextFile(strFileOuput) End If objFile.WriteLine strBufferNew objFile.Close Set objFile = Nothing Set objFSO = Nothing
Aptly saved the file as searchandreplace.vbs
Later found out that we need a singly rooted XML file:
Error message via LogParser:
A couple of other important pieces of business are:
- Needed ^ in front of special characters such as when using the echo commands (http://www.robvanderwoude.com/useless.php#EscapeChar)
- When we tried accessing the file via the log Parser encountered an error stating “Error Parsing Query – Invalid at the top level of the document. [Unknown Error]”.Tried to load via Chrome and encountered error stating “Error on line NNN at column 1: Extra content at the end of the document”.Using Notepad++, found a misplaced SUB at the bottom of the file. The error was traced back to standard file merging via copy command – Changed to binary merging by adding a /B argument to copy command.
Errors Encountered (in logParser):
- Error parsing query: Error loading : Invalid at the top level of the document. [Unknown Error]
- Errors Encountered (Chrome Browser):
- Error : Extra content at the end of the document
- Errors Encountered (Log file Contents):
- In echo command, prefixed special characters (< and >) with ^
- In copy command, augmented copy command with /B
REM -------------------------------------------------------------------------------------------------------------------------- @echo off cscript searchandreplace.vbs app-log4net.xml app-log4net-proper.xml echo ^<log4net^> > header.txt echo ^</log4net^> > footer.txt copy /B header.txt + app-log4net-proper.xml + footer.txt app-log4net-proper_v2.xml REM --------------------------------------------------------------------------------------------------------------------------
- Practical .Net – Debugging Tools for the .Net Developer (Peter Vogel)
- Lizard-Labs.Net — Using Regular Expression and log4net input formats with Log Parser Lizard
- Hey Scripting Guys – How can and I find and Replace Text in a Text file?
- Apache log4net – Manual Configuration
- How to write data to a file with OpenTextFile
- SAS – Format modifiers