Coolthing Of Theday

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Tuesday, 12 November 2013

It's Log Parser Day! Robert Sheldon shows how Log Parser can be used for ETL

Posted on 17:11 by Unknown

SQL Home - SQL Tools - Microsoft's Log Parser Utility: Swell ETL

For loading text, CSV or XML files into SQL Server, the Log Parser utility, with its amazing SQL engine,  is likely to be the obvious choice. Although initially developed purely for converting IIS logs, the Log Parser can turn its hand to a range of formats including even event logs or the windows registry.  

First off, Microsoft’s Log Parser utility is not a SQL Server tool. Log Parser is a powerful Windows command-line utility that can extract data from a variety of sources—IIS logs, XML and CSV files, Active Directory objects, Network Monitor capture files, and the Windows registry, to name a few—and output the data to various files and systems, most notably SQL Server. In fact, Log Parser makes importing data into a SQL Server database so simple, you’ll wonder why you haven’t been using the tool all along.

At the core of the Log Parser utility is a “SQL-like” engine that processes data as it’s retrieved from the source and sent to the destination. You can think of Log Parser as a mini extract, transform, and load (ETL) application that uses input formats to extract data from its source and output formats to send the data to its destination.

An input format provides the source data to the engine as a record set, similar to the way rows are stored in a table. Each input format serves as a record provider specific to the source from which the data is retrieved. For example, you would use the xml input provider to retrieve data from an XML file.

Output formats also present the processed information as record data, with each output format specific to the target destination type. If you were sending data to a SQL Server database, for instance, you would use the sql output format.

For details about the available input and output formats supported by Log Parser, as well as information about other features, see the Log Parser help file (LogParser.chm). The file is added to the directory where Log Parser is installed when you do a complete installation or you include the documentation component as part of a custom installation. You can download Log Parser from the Microsoft Download Center. There you will also find installation instructions.

Using Log Parser to retrieve data

Log Parser comes in two versions: a command-line executable and a DLL containing COM objects that applications can use to run Log Parser operations. This article focuses on the command-line utility and how you can use it to import data into a SQL Server database.

...

Working with Log Parser

The examples I’ve shown you in this article have all retrieved data from the System event log, but you’re certainly not limited to that log. You can retrieve data from other event logs, multiple logs, and a variety of other sources, such as Active Directory, the registry, IIS logs, text files, or information about the file directory itself. Log Parser is a flexible and powerful tool that can be useful in a variety of circumstances. And because of the utility’s “SQL-like” logic, most of the data you can retrieve through Log Parser can be saved to a SQL Server database. If you can write a T-SQL SELECT statement, you can use Log Parser to store all sorts of information in your SQL Server databases.

imageimage

Robert's post is actually a great, in-depth post on one more more useful yet overlooked utilities, Log Parser...

And anyway, it's been nearly a month since the last Log Parser post! You can't expect me to hold off forever, do you? :P

 

Related Past Post XRef:
PIE! (charts) - Log Parser and the Office Web Components together turns your logs into pie charts and more..

Log Parser Studio 2.0 now out (Log Parser GUI++)
Log Parser Studio - Think "Log Parser GUI" Or "Making Log Parser click-click fun and easy to use..." or "Query Analyzer for Log Parser"

Learning Log Parser Studio in two parts... (From Install to Library Ninja)

Here's a look at the Microsoft Log Parser from a different point of view, from the Computer Forensics' side of the house OR Check out a 'Query Analyzer/SSMS' for Log Parser called Log Parser Lizard

Log Parser Ping Graph Fun (aka “Using Log Parser to parse command line output”)
SELECT * FROM Log... with the cool tool that’s been around for years, Log Parser!

Download details: Log Parser 2.2

The Unofficial Log Parser Support Site
IIS Diagnostics Toolkit (January 2006)
SQL Server 2000 Report Pack for IIS Logs

Email ThisBlogThis!Share to XShare to FacebookShare to Pinterest
Posted in SystemAdministration, Utility, Windows | No comments
Newer Post Older Post Home

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Popular Posts

  • Mr. 7,000! This is my 7,000th post...
    Before this post; After; 20 visits between taking these snaps? Oh wait, that's probably me searching for past related posts....
  • "Windows Server Essentials Media Pack" (DNLA Stream, HTML5 and Dashboard Media stuff)
    Microsoft Downloads - Windows Server Essentials Media Pack This pack enables the media streaming functionality for Windows Server 2012...
  • Rad Gate Post... Get your Red Gate Post here...
    simple talk - Melanie Townsend - Get a copy of the Red Gate Post We recently put together a newspaper of some of the best articles fr...
  • Windows Management Framework 4.0 (PowerShell 4, PowerShell ISE, Management OData, WMI, etc.) now available
    Keith Hill's Blog - PowerShell 4.0 Now Available You can get PowerShell 4.0 for down level operating systems now via the WMF 4.0 d...
  • Viasfora - Your new favorite Visual Studio Text/*ML Editing Extension?
    Winterdom - Introducing Viasfora A couple of days ago, I unveiled Viasfora , my latest attempt at building a decently packaged extensi...
  • "Windows Server [2012 R2]: The Best Infrastructure to Run Linux Workloads"
    In the Cloud - What’s New in 2012 R2: Enabling Open Source Software Part 4 of a 9-part series . ... There are a lot of great s...
  • [Hardware Review] Life with Haswell... Haswell/Harris Beach Intel SDS Ultrabook Review - Part 2
    "So Greg, how's life with Haswell been?" "Pretty Sweet! (Mostly)" I've been given an opportunity to review t...
  • Fuzzy Lookup Add-In for Excel (Insert lame "Fuzzy, wuzzy was an Excel..." snip here)
    Microsoft Downloads - Fuzzy Lookup Add-In for Excel The Fuzzy Lookup Add-In for Excel performs fuzzy matching of textual data in Exce...
  • Caliburn.Micro v1.5.0 released (CM gets Tasks, Async/Await and Share/Setting for RT... and bug fixes of course)
    Caliburn.Micro - Caliburn.Micro v1.5.0 "Release Notes This release fixes many bugs. It also adds support for Task and async/a...
  • Just about everything you ever wanted to know about SQL Server Date and Time Data Types...
    CodeProject - Date and Time Data Types and Functions - SQL Server (2000, 2005, 2008, 2008 R2, 2012) Introduction It would be bette...

Categories

  • .Net
  • 3DPrinting
  • AFeedYouShouldRead
  • Agile
  • ALM
  • Amazon
  • Amiga
  • Analytics
  • Android
  • ASP.NET
  • Azure
  • BigData
  • bing
  • Blogging
  • Book
  • BookReview
  • BUILD
  • C
  • C#
  • C++
  • Career
  • Cat
  • cheatsheet
  • ClickOnce
  • Cloud
  • ComputerHardware
  • css
  • Data
  • DBA
  • DependencyInjection
  • Deployment
  • Design
  • Development
  • devops
  • DVCS
  • ebook
  • EDD
  • Education
  • EnterpriseLibrary
  • EntityFramework
  • Exchange
  • Expression
  • gadget
  • Game
  • GIT
  • Google
  • Government
  • Hadoop
  • hardware
  • HardwareReview
  • HaswellReview
  • HTML5
  • Humor
  • IE
  • IEExtension
  • IfAllElseFails
  • IIS
  • ILMerge
  • Image
  • Infographic
  • interview
  • InversionOfControl
  • Java
  • Javascript
  • Kinect
  • LightSwitch
  • LINQ
  • Linux
  • LosAngeles
  • Lucene
  • Lync
  • MEF
  • Metro
  • MicrosoftOffice
  • MicrosoftOutlook
  • Mono
  • MVC
  • MVVM
  • NetMon
  • NLP
  • NoSQL
  • NuGet
  • OData
  • OneNote
  • OpenXML
  • Paint.Net
  • Personal
  • Photosynth
  • Physics
  • portable
  • Poster
  • PowerShell
  • Preparedness
  • Presentation
  • Prism
  • PrivateCloud
  • RegEx
  • RemoteDesktop
  • Reporting
  • RIAServices
  • Science
  • ScienceFiction
  • Scratch
  • Scrum
  • ServiceBus
  • SharePoint
  • Silverlight
  • SimiValley
  • SPA
  • Space
  • SQLServer
  • Storyboard
  • Surface
  • SVG
  • SystemAdministration
  • T4
  • TeamBuild
  • TeamFoundationServer
  • TechEd
  • Training
  • TypeScript
  • UnitTesting
  • UnityApplicationBlock
  • Utility
  • Veteran
  • VirtualMachine
  • Visio
  • VisualBasic
  • VisualStudio
  • WCF
  • Web X.X
  • Webcast
  • WebFeed
  • WebMatrix
  • Windows
  • Windows7
  • Windows8
  • Windows8.1
  • WindowsHomeServer
  • WindowsLiveWriter
  • WindowsPhone
  • WindowsServer
  • WinRT
  • WiX
  • WMI
  • WOPI
  • WPF
  • XAML
  • XBox360
  • XboxOne
  • zombie

Blog Archive

  • ▼  2013 (500)
    • ►  December (12)
    • ▼  November (61)
      • Red Gate SSMS Ecosystem - The free SQL Server Mana...
      • Two SQL Server Resources that you might want to ta...
      • [Kickstarter of the Day]Making your paper airplane...
      • Infographic for the English Grammar Challenged...
      • Taking File Explorer to 11! (Okay 10... ) - 10 Tip...
      • Creating Per-Monitor DPI-Aware Applications Code S...
      • Windows 8.1 Store App Paper Prototyping Templates
      • Having your Amiga and Pi to... Using Raspberry Pi ...
      • [Humor] Death by Cube...
      • Happy VM Day! The Visual Studio 2013 RTM ALM Virtu...
      • Being smart when opening SkyDrive Smart Files in D...
      • If you're not cheating, you're not... using these ...
      • No If's here.. Jason Haley Web Appifys his Interes...
      • Viasfora - Your new favorite Visual Studio Text/*M...
      • Battling the Console Wars, one mini-3d print job a...
      • [Book Review] 'LÖVE for Lua Game Programming'
      • The TMI Infographic of the day... aka the world's ...
      • Windows Azure Learning Resource Link Round-up
      • Opening EntLib - The Microsoft Enterprise Library ...
      • Office/Exchange File Format,Specification and Prot...
      • Pst... Storing PST's on a network share? Still a n...
      • cough... cough... feeling a little [Xbox One deliv...
      • Surface Pro / Pro 2 Battery Life Extension Tip... ...
      • 1st Annual Socaltech 50 - 50 SoCal Tech companies ...
      • A test lab to play with eDiscovery across Exchange...
      • A word or two or 10 about Word Clouds
      • "Developer's Guide to Microsoft Enterprise Library...
      • New Amiga motherboard, updated AmigaOS... Who says...
      • 10 Professionals, 10 views on the coming trends in...
      • GQL, no, not the Gnome Query Language, the Genome ...
      • Working Office Remote (No that kind of remote...)....
      • [Book Review - Preview] 'LÖVE for Lua Game Program...
      • Nokia Imaging SDK v1.0 RTW now available, official...
      • New from NirSoft - WhoIsConnected[to my darn netwo...
      • Kodu Game Lab v1.4.1.0
      • "Community [e]Book of PowerShell Practices" Septem...
      • Katniss the cat in the kitty remake of The Hunger ...
      • Get your own California Ghost Town, Craigslist'd f...
      • VS2013 powers up with the new and updated Producti...
      • Apple II DOS source code available. All you have t...
      • Jumping into SQL Server 2014 with these two Micros...
      • Amazon goes virtual...desktop.. with their new Ama...
      • Missed Today's Visual Studio 2013 Launch (and ther...
      • "The Field Guide to Data Science" Free eBook of th...
      • Wax poetic with this new WiX Setup Project Editor,...
      • Whoa there's allot of the free NOAA [resources]
      • It's Log Parser Day! Robert Sheldon shows how Log ...
      • Visual Studio / Team Explorer 2013 no longer requi...
      • Preparing Patriotic Presentations with PowerPoint ...
      • Thank you for your thank you's, a veteran's day note
      • Don't Present, Resonate - Nancy Duarte's resonate ...
      • Dave McKinstry's Massive Missive of More VS 2013 L...
      • Drawing for Developers (Yes, you can!)
      • So tasty you'll want to eat your phone, Marmalade ...
      • We're from MSDN Magazine and we're here to help......
      • These beautiful D&D maps make you want to break ou...
      • Write a killer job description so you don't have t...
      • In with the new for the old school, the NNTP bridg...
      • "Windows 8.1 Quick [keyword] Guide for Business(?)"
      • Cloud VDI seems to be the new Dev desktop shiny, s...
      • "hackathon-in-a-box.org" is you're, well, Hackatho...
    • ►  October (65)
    • ►  September (38)
    • ►  August (47)
    • ►  July (75)
    • ►  June (39)
    • ►  May (40)
    • ►  April (42)
    • ►  March (39)
    • ►  February (42)
Powered by Blogger.

About Me

Unknown
View my complete profile