Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: preg_replace(): Compilation failed: invalid range in character class at offset 4 in /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php on line 340 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 Warning: Cannot modify header information - headers already sent by (output started at /home/.sites/946/site2164767/web/blog/wp-content/plugins/crayon-syntax-highlighter/crayon_langs.class.php:340) in /home/.sites/946/site2164767/web/blog/wp-includes/rest-api/class-wp-rest-server.php on line 1723 {"id":103,"date":"2016-03-09T21:53:46","date_gmt":"2016-03-09T19:53:46","guid":{"rendered":"http:\/\/blog.zoolway.com\/?p=103"},"modified":"2016-03-09T22:05:07","modified_gmt":"2016-03-09T20:05:07","slug":"logging-human-readable-vs-machine-parsable","status":"publish","type":"post","link":"http:\/\/blog.zoolway.com\/logging-human-readable-vs-machine-parsable","title":{"rendered":"Logging – Human Readable vs. Machine Parsable"},"content":{"rendered":"

I have often encouraged other to implement proper logging even in the smallest projects. My .NET\u00a0MVVM template<\/a> available at GitHub comes with log4net per default. There is no need to reinvent the wheel. Besides log4net there is NLog and other frameworks. Most can target many diverse storages like files (with automatic rolling), the Windows event log, UDP ports and more. This way we get some decent log files.<\/p>\n

Human Readable<\/h2>\n

With some log file rolling and different log levels it is mostly a good idea to log\u00a0data which can be important, especially in production debugging. When you get the users to send you the log file or a feedback tool does that for them, great. Now you got your hands on some information which should look useful. Example:<\/p>\n

2016-03-08 22:30:53,263 [8] DEBUG Zw.JsonLogViewer.AppBootstrapper - Creating bootstrapper\r\n2016-03-08 22:30:53,717 [8] INFO  Zw.JsonLogViewer.ViewModels.ShellViewModel - Loading last-used logfile: 'D:\\Daten\\Tech Files\\Development\\dotNET\\..\\bin\\Debug\\1\\lastrun.json.log'\r\n2016-03-08 22:30:53,729 [11] DEBUG Zw.JsonLogViewer.ViewModels.LogViewModel - Opening log 'D:\\Daten\\Tech Files\\Development\\dotNET\\..\\bin\\Debug\\1\\lastrun.json.log'\r\n2016-03-08 22:30:53,737 [11] DEBUG Zw.JsonLogViewer.Parsing.Parser - Parsing log file 'D:\\Daten\\Tech Files\\Development\\dotNET\\..\\bin\\Debug\\1\\lastrun.json.log'\r\n2016-03-08 22:30:53,859 [11] DEBUG Zw.JsonLogViewer.Parsing.Parser - Parsed 331 lines into 331 log entries\r\n2016-03-08 22:30:53,860 [11] DEBUG Zw.JsonLogViewer.Parsing.Parser - Providing 6 keys as columns: date,level,appname,logger,thread,message<\/pre>\n
Write done what is important in a nice human readable way. No binary dumps needed \ud83d\ude09<\/p>\n
Okay, now that you got this 8MB of text right infront of you, the text editor for your choice will search for “ERROR”, a logger name and other subjects. Some advanced editors are able to hide all lines in a file which do not<\/em> match your criteria. This way you can try to only show INFO messages.<\/p>\n
When you have done this, you will find out that handling logs with several hundreds or even more lines can be a pain. Although these files are there to help you finding bugs – and might be your only debugging help in production scenarios (where\u00a0it might be not possible to connect with a remote debugger).<\/p>\n
Machine Parsable<\/h2>\n
Of course, I do not want to go back to any kind of proprietary log format – less readable and you have to write your own parser. A good idea is to go with a well know format. XML got infamous because most people do not like XML namespaces and the redundant information due to the closing tags. The fall of XML was the rise of JSON. It is a more lightweighted format (also there is an optional schema feature out there) and parsers are available for almost any programming language. If we provide the information about our log statements as JSON objects it could easily be parsed.<\/p>\n
{\"date\":\"2016-03-06T18:18:37.3018646+01:00\",\"level\":\"INFO\",\"logger\":\"State\",\"thread\":\"9\",\"message\":\"Beta 'Anzu' 2.2.1.3 starting\"}\r\n{\"date\":\"2016-03-06T18:18:37.4539737+01:00\",\"level\":\"DEBUG\",\"logger\":\"MyApp.App\",\"thread\":\"9\",\"message\":\"This log channel is DEBUG enabled and might contain really much information. For production usage we recommend setting the log level to INFO or WARN.\"}\r\n{\"date\":\"2016-03-06T18:18:37.5140168+01:00\",\"level\":\"INFO\",\"logger\":\"MyApp.AppBootstrapper\",\"thread\":\"9\",\"message\":\"Creating bootstrapper...\"}\r\n{\"date\":\"2016-03-06T18:18:37.5820649+01:00\",\"level\":\"DEBUG\",\"logger\":\"MyApp.AppBootstrapper\",\"thread\":\"9\",\"message\":\"Running bootstrapper configuration\"}\r\n{\"date\":\"2016-03-06T18:18:37.7261679+01:00\",\"level\":\"DEBUG\",\"logger\":\"MyApp.SysCore.License\",\"thread\":\"9\",\"message\":\"Providing empty license without translation\"}\r\n{\"date\":\"2016-03-06T18:18:37.7411796+01:00\",\"level\":\"DEBUG\",\"logger\":\"MyApp.App\",\"thread\":\"9\",\"message\":\"Application is starting...\"}<\/pre>\n
As you can see this approach assumes every file line is a JSON object. You could also have the complete file to be an array of JSON objects – but this would require to rewrite the end of the file all the time to enlarge the array. If the app crashed and the log writer cannot flush, you might end up with a destoryed JSON array. Therefore I prefer the “one line = one JSON object” approach.<\/p>\n
You can image, this is slightly harder to read for a human but not too hard. Also using a text editor’s search feature might work even better because of the object property names visible all the time.<\/p>\n
A format like this can be achieved in log4net with the additional libraries log4net.Ext.Json,\u00a0log4net.Ext.Json.Serializers.Newtonsoft and (of course) Json.NET (former Newtonsoft.JSON, no .NET app using JSON without it). This is a rolling file appender configuration example:<\/p>\n
<appender name=\"RollingFileJson\" type=\"log4net.Appender.RollingFileAppender\">\r\n  <file value=\"lastrun.json.log\"\/>\r\n  <appendToFile value=\"false\"\/>\r\n  <rollingStyle value=\"Size\"\/>\r\n  <maxSizeRollBackups value=\"5\"\/>\r\n  <maximumFileSize value=\"10MB\"\/>\r\n  <layout type=\"log4net.Layout.SerializedLayout, log4net.Ext.Json\">\r\n    <decorator type=\"log4net.Ext.Json.Serializers.Newtonsoft.NewtonsoftDecorator, log4net.Ext.Json.Serializers.Newtonsoft\" \/>\r\n    <renderer type=\"log4net.ObjectRenderer.JsonObjectRenderer, log4net.Ext.Json\">\r\n      <factory type=\"log4net.Ext.Json.Serializers.Newtonsoft.NewtonsoftFactory, log4net.Ext.Json.Serializers.Newtonsoft\" \/>\r\n    <\/renderer>\r\n    <default \/>\r\n    <remove value=\"ndc\" \/>\r\n    <remove value=\"appname\" \/>\r\n  <\/layout>\r\n<\/appender><\/pre>\n
This is a lot of assemblies but you might use Json.NET anyways and the other two are due to the modularity. And you could still add your own renderer.<\/p>\n
With all this great machine-readable output there is one question left.<\/p>\n
How can I read it with filtering and sorting?<\/h3>\n
I was suprised when I did not find \u00a0a nice little tool which would parse any kind of JSON objects into a column based view. If you can recommend any tool for this I would highly appreciate any comment to this post!<\/p>\n
What I was looking for was a tool where\u00a0not all lines would require the same JSON object. There could be a property only added to specific log lines for example. The tool should not crash neither ignore an inconsist JSON object layout through the log file.<\/p>\n
Zw.JsonLogViewer<\/h2>\n
Did I say it would be easy to parse log entries in JSON\u00a0format? Yes, it is.\u00a0So I built a small MVVM application to open those files and be able to search, filter and sort log entries.<\/p>\n
\"zwjsonlogviewer\"<\/a><\/p>\n
This small sample app is available at GitHub<\/a>\u00a0as open source under the Apache 2 license.<\/p>\n
It currently features:<\/p>\n