Coolthing Of Theday

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Monday, 22 April 2013

Example using the Task Parallel Dataflow Library help keep... well, your data, um, flowing...

Posted on 08:58 by Unknown

.Net Curry - Building an Image Resizer using .NET Parallel Dataflow Library in .NET 4.5

Abstract: With .NET 4.5, the Task Parallel Library team went another step ahead and built a little known library called Parallel Dataflow Library. This article explains why as a .NET developer, you must know about this powerful library.

By now, we’ve all heard about the Task Parallel Library (TPL) introduced in .NET 4.0. TPL is really neat and goes a long way in making parallel programming palatable for a bigger slice of developers. With .NET 4.5, the TPL team actually went another step ahead and built a little known library called Parallel Dataflow Library.

Parallel what… you ask?

Yeah well, the naming isn’t very helpful and nor has it been blogged about much! But the Parallel Dataflow library is actually quite powerful. It leverages the TPL to provide us with a nice in-process, message passing functionality. This helps in creating a ‘pipeline’ of ‘tasks’ when we are trying to build asynchronous applications. Confused? Let’s try with an example.

Let’s think of this scenario! You need to process multiple images (some processing resizing/applying Instagram like filters or something else, and then saving the new images). Also the images are going to come into your system intermittently. Once they arrive, you need to process them and put them at a destination location. How shall we go about it?

Well one way is to wait for images to arrive, when they arrive (either one at a time or in a bulk), pick each available image, load it, process it, move it to output and go back to waiting. We can do this in two ways

image

This as we can see is a monolithic and synchronous action for three potentially independent processes. For every file we are doing these three steps while the next files waits to be considered.

In the second approach, once a file is loaded, the ‘loader’ can handover the image to the ‘processor’ and go back to loading again! Similarly, once a processor has done processing, it can handover the file to the output system to save and go back to processing the next file, ditto with the output processor! So these three tasks can actually be piped together, on the input side we have file or files as they come in, on the output side you have file as it gets done. The above image would get transformed into the following

image

...

As we can see, it’s a pipeline of smaller tasks that communicate with each other when data is flowing through time. The Parallel Dataflow Library helps us build these kind of solutions with relative ease. The parallel execution and asynchronous behavior is taken care of by the TPL, on which the Dataflow library is build.

Now that we know what it does, let’s see some of the types and structures that help the Parallel Dataflow Library achieve this.

Parallel Dataflow Library – Programming Model

Continuing with our above example, if we look at the three blocks we have, the first one can be considered a Source Block, the Last one can be considered the target block and the center one can be considered the transformation block that moves data from source to target while doing some action with it.

...

Conclusion

That was a very quick and brief introduction to the Parallel Dataflow Library of TPL! It is quite extensive with respect to the Blocks that it supports and it’s a matter of cooking up the use case we want to throw at it. I hope to have at least setup the premises under which such an architecture should be considered.

Overall the Dataflow library helps us do ‘in-process’ message passing that can be used for dataflow and pipelining at a much higher level as in with objects in managed code, as opposed to with bits and bytes inside of CPU execution pipelines! "

If you have data or files that flow from one state to a next and have a bunch of them and want or need to live in a multithreaded world (and who doesn't these days?), you might want to check out the Task Parallel Dataflow library. I mean it's there already, you might was well, right?

 

Related Past Post XRef:
Primer for TPL TDF (Task Parallel Library- Task Parallel Dataflow)
TPL (Task Parallel Library) Dataflow CTP Released
PDC10 Async Round-up

Email ThisBlogThis!Share to XShare to FacebookShare to Pinterest
Posted in .Net, Development | No comments
Newer Post Older Post Home

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Popular Posts

  • Mr. 7,000! This is my 7,000th post...
    Before this post; After; 20 visits between taking these snaps? Oh wait, that's probably me searching for past related posts....
  • "Windows Server Essentials Media Pack" (DNLA Stream, HTML5 and Dashboard Media stuff)
    Microsoft Downloads - Windows Server Essentials Media Pack This pack enables the media streaming functionality for Windows Server 2012...
  • Rad Gate Post... Get your Red Gate Post here...
    simple talk - Melanie Townsend - Get a copy of the Red Gate Post We recently put together a newspaper of some of the best articles fr...
  • Windows Management Framework 4.0 (PowerShell 4, PowerShell ISE, Management OData, WMI, etc.) now available
    Keith Hill's Blog - PowerShell 4.0 Now Available You can get PowerShell 4.0 for down level operating systems now via the WMF 4.0 d...
  • Viasfora - Your new favorite Visual Studio Text/*ML Editing Extension?
    Winterdom - Introducing Viasfora A couple of days ago, I unveiled Viasfora , my latest attempt at building a decently packaged extensi...
  • "Windows Server [2012 R2]: The Best Infrastructure to Run Linux Workloads"
    In the Cloud - What’s New in 2012 R2: Enabling Open Source Software Part 4 of a 9-part series . ... There are a lot of great s...
  • [Hardware Review] Life with Haswell... Haswell/Harris Beach Intel SDS Ultrabook Review - Part 2
    "So Greg, how's life with Haswell been?" "Pretty Sweet! (Mostly)" I've been given an opportunity to review t...
  • Fuzzy Lookup Add-In for Excel (Insert lame "Fuzzy, wuzzy was an Excel..." snip here)
    Microsoft Downloads - Fuzzy Lookup Add-In for Excel The Fuzzy Lookup Add-In for Excel performs fuzzy matching of textual data in Exce...
  • Caliburn.Micro v1.5.0 released (CM gets Tasks, Async/Await and Share/Setting for RT... and bug fixes of course)
    Caliburn.Micro - Caliburn.Micro v1.5.0 "Release Notes This release fixes many bugs. It also adds support for Task and async/a...
  • Just about everything you ever wanted to know about SQL Server Date and Time Data Types...
    CodeProject - Date and Time Data Types and Functions - SQL Server (2000, 2005, 2008, 2008 R2, 2012) Introduction It would be bette...

Categories

  • .Net
  • 3DPrinting
  • AFeedYouShouldRead
  • Agile
  • ALM
  • Amazon
  • Amiga
  • Analytics
  • Android
  • ASP.NET
  • Azure
  • BigData
  • bing
  • Blogging
  • Book
  • BookReview
  • BUILD
  • C
  • C#
  • C++
  • Career
  • Cat
  • cheatsheet
  • ClickOnce
  • Cloud
  • ComputerHardware
  • css
  • Data
  • DBA
  • DependencyInjection
  • Deployment
  • Design
  • Development
  • devops
  • DVCS
  • ebook
  • EDD
  • Education
  • EnterpriseLibrary
  • EntityFramework
  • Exchange
  • Expression
  • gadget
  • Game
  • GIT
  • Google
  • Government
  • Hadoop
  • hardware
  • HardwareReview
  • HaswellReview
  • HTML5
  • Humor
  • IE
  • IEExtension
  • IfAllElseFails
  • IIS
  • ILMerge
  • Image
  • Infographic
  • interview
  • InversionOfControl
  • Java
  • Javascript
  • Kinect
  • LightSwitch
  • LINQ
  • Linux
  • LosAngeles
  • Lucene
  • Lync
  • MEF
  • Metro
  • MicrosoftOffice
  • MicrosoftOutlook
  • Mono
  • MVC
  • MVVM
  • NetMon
  • NLP
  • NoSQL
  • NuGet
  • OData
  • OneNote
  • OpenXML
  • Paint.Net
  • Personal
  • Photosynth
  • Physics
  • portable
  • Poster
  • PowerShell
  • Preparedness
  • Presentation
  • Prism
  • PrivateCloud
  • RegEx
  • RemoteDesktop
  • Reporting
  • RIAServices
  • Science
  • ScienceFiction
  • Scratch
  • Scrum
  • ServiceBus
  • SharePoint
  • Silverlight
  • SimiValley
  • SPA
  • Space
  • SQLServer
  • Storyboard
  • Surface
  • SVG
  • SystemAdministration
  • T4
  • TeamBuild
  • TeamFoundationServer
  • TechEd
  • Training
  • TypeScript
  • UnitTesting
  • UnityApplicationBlock
  • Utility
  • Veteran
  • VirtualMachine
  • Visio
  • VisualBasic
  • VisualStudio
  • WCF
  • Web X.X
  • Webcast
  • WebFeed
  • WebMatrix
  • Windows
  • Windows7
  • Windows8
  • Windows8.1
  • WindowsHomeServer
  • WindowsLiveWriter
  • WindowsPhone
  • WindowsServer
  • WinRT
  • WiX
  • WMI
  • WOPI
  • WPF
  • XAML
  • XBox360
  • XboxOne
  • zombie

Blog Archive

  • ▼  2013 (500)
    • ►  December (12)
    • ►  November (61)
    • ►  October (65)
    • ►  September (38)
    • ►  August (47)
    • ►  July (75)
    • ►  June (39)
    • ►  May (40)
    • ▼  April (42)
      • Unusual Microsoft Download of the Day: "Cancelling...
      • Enterprise Library 6 and Unity 3 are out today... ...
      • NuGet your Enterprise?
      • Infragistics Releases 13.1 with new set of Windows...
      • Getting into the mood... (well...) Microsoft Resea...
      • Virtual CD-ROM Control Panel download - A granddad...
      • Public Sector Developer Weblog highlights a whole ...
      • The First [12] Steps in using Caliburn Micro to bu...
      • You are such a Code Digger... Code Digger VS2012 e...
      • I trace, you trace, we all IntelliTrace (or want t...
      • Example using the Task Parallel Dataflow Library h...
      • Learning Log Parser Studio in two parts... (From I...
      • "The Essential Binary Repository Management Cheat ...
      • Bye bye Kona code name... Hello "Prism for Windows...
      • 14 Minute cartooned video guide to to building you...
      • Just about everything you ever wanted to know abou...
      • TFS/Visual Studio 2012 Update 3 CTP 1 is available...
      • A view into the creation of NASA'a spaceappschalle...
      • The final final definitive (for now) answer for Wi...
      • Taking Facial Detection to the next level (and ful...
      • DevOps & IntelliTrace, better than beer and pizza ...
      • [Humor] "How to make an awesome illustration [for ...
      • What do Fiddler, LinqPad, Excel and SharePoint hav...
      • What would the Xbox 360 “Achievement Unlocked” sou...
      • Unity vs. MEF - right for you, one is...
      • Learning, living, being SQL Server for Dev's video...
      • Going with the GeoFlow for Excel 2013... Free 3D v...
      • Office Web Apps Server isn't just for SharePoint.....
      • "The Last Visibility Converter" (you'll ever need)
      • Microsoft Script Explorer - So long and thanks for...
      • Two Firsts, one great taste... A "Database-First C...
      • Community AdventureWorks on Azure one year later, ...
      • Powering your Zipping with PowerShell and 7-Zip (a...
      • WPF is to PowerShell as the PowerShell ISE is to E...
      • Nothing like a little LinqPad fun for a Friday - "...
      • DLM on MSDN - Database Lifecycle Management curate...
      • Visual Studio 2012 Update 2 is RTW... (and it's a ...
      • Dev'ing for Ops - How to create System Center Exte...
      • Scrum Primer, the v2...
      • Now that's Qool
      • Sql Server info tip of the day: Alter Columns - Kn...
      • Open Source NFC Emulator for Windows Phone 8 with ...
    • ►  March (39)
    • ►  February (42)
Powered by Blogger.

About Me

Unknown
View my complete profile