Libraries Under the Hood

By now, you should have a better understanding of what Windows 7 libraries represent and how they can benefit users as well as developers. This is the third post in a series of Windows 7 Libraries posts. In the previous post, Understanding Windows 7 Libraries, we explained what libraries are and the important role they play in users’ day-to-day work. In this post, we explain the underlying architecture that supports libraries.

As usual, let’s start with a quick historic review of Libraries' ancestors. Windows Vista introduced a new storage scenario and a new user profile namespace called the “Known Folders.” This represented the next evolutionary step from “standard folders” that were based on a constant special item ID list (CSIDL). The CSIDL values provide a system-independent way to identify special folders used frequently by applications. In Windows Vista, we replaced these values with the Known Folders system where a KNOWNFOLDERID constant identifies each special folder.

Both CSIDL and Known Folders included support for users’ specific content types such as Document, Music, Pictures and Videos – these folders are part of the user storage profile. In Windows 7, Libraries extend some of the common user Known Folders, like My Documents, into the Documents Library. If you review the KNOWNFOLDERID in Windows 7 Beta SDK, you will find that the default Libraries are Known Folders themselves, which make sense since we want a unified programmatic way to access user’s content. By default, each Library contains two physical file locations:

  • The user's personal folder by type (Documents, Pictures, Video…), which is represented by the default Known Folder that is included in a Library (that is, My Documents in the Documents Library), and it is also the default save location.
  • The public folder by type

For custom Libraries, the default save location is the first folder added.

In some ways, a Library is similar to a folder. From a user's point of view, a library looks and behaves just like a regular folder. However, unlike a folder, a Library gathers files that are stored in several locations into a single view. This is a subtle, but important difference. Libraries don't actually store users’ items, they only monitor folders that contain items, and let users access and arrange these items in different ways. They also let users and developers access the list of folders and their contents. For example, if you try to save a text document (from Notepad) directly to the Documents Library, the file will be saved. However, we just said that Libraries are NOT regular folders that contain files, so what’s going on? Well, the text file is saved to the Documents Library default save location, which is, by default, the user’s Documents folder – also known as “My Documents” and is part of the user profile. The user has full control over the default save location as well as the other locations that are part of a library. As shown in the next figure, the Picture Library management dialog (which is accessible directly from the Windows Explorer UI) contains several locations on my local hard drive with one of the folders designated as the default save location.

Library Management Dialog

It is important to note that the Pictures Library didn’t replace the My Pictures folder from Windows Vista; it simply includes that folder in the Pictures library as one of many locations. Also, please note that the CSIDL system and its APIs system are still supported for application compatibility. However, we do not recommend that you use them in new application development.We recommend that you use the known folder system instead

A Library in Windows 7 is stored as an XML definition file that has a file extension of “.library-ms.” The file name is the actual library’s name. For example, the Documents Library is represented by an XML file called Documents.library-ms. Library descriptions are saved on disk in the %appdata% Roaming\Microsoft\Windows\Libraries folder (also known as FOLDERID_Libraries). You can use the Library definition files to monitor changes to the library as we will explain in future posts

Let’s dig into the Documents Library definition file schema. The XML structure is pretty self-explanatory, but let’s explain a few of its elements. At the top of the file, we can find the Library “header” information:

<libraryDescription xmlns="http://schemas.microsoft.com/windows/2009/library">
<name>@shell32.dll,-34575</name>
<ownerSID>S-1-5-21-2127521184-1604012920-1887927527-4897363</ownerSID>
<version>4</version>
<isLibraryPinned>true</isLibraryPinned>
<iconReference>imageres.dll,-1002</iconReference>
<templateInfo>
<folderType>{7D49D726-3C21-4F05-99AA-FDC2C9474656}</folderType>
</templateInfo>

The root XML element is libraryDescription, which contains child elements that define the library:

  • <ownerSID> is an element that defines the Security ID of the user who created this library to isolate Libraries and protect user data from other users
  • <isLibraryPinned> is a Boolean element that defines if the library is pinned to the left navigation pane in Windows Explorer and NOT to the Taskbar
  • <version> defines the content version of this library, which reflects the number of times the library definition file has been changed
  • <templateInfo> is an optional container element that lets the author specify the folder type (Documents, Pictures, Video) for controlling the arrangements of views in Windows Explorer
  • <iconReference> defines an icon resource using standard Windows Shell resource style; for example: <iconReference> C:\Windows\system32\imageres.dll,-65 </iconReference>. This icon presents the library in the Windows Explorer

Users can’t change the default libraries icon via Windows Explorer, nor can they assign the icon to a new customized user-defined library. However, this can be done programmatically using the API, which we will cover in future posts.

A key part of the XML is the list of locations that the Library represents:

<searchConnectorDescriptionList>
<searchConnectorDescription publisher="Microsoft" product="Windows">
<description>@shell32.dll,-34577</description>
<isDefaultSaveLocation>true</isDefaultSaveLocation>
<isSupported>true</isSupported>
<simpleLocation>
<url>knownfolder:{FDD39AD0-238F-46AF-ADB4-6C85480369C7}</url>
<serialized>MBAAAE…. </serialized>
</simpleLocation>
</searchConnectorDescription>
  • <searchConnectorDescriptionList> contains one or more search connectors that map to physical locations included in this library
  • <searchConnectorDescription> contains a <simpleLocation> element that describes one location included in the library
  • <url> defines a URL for this location and it is there for human-readability only, this cannot be used by developers since it is likely to be out of date.
  • <serialized> this is the actual representation (Base64) of a location in a library which is as a serialized ShellLink object

One final note - Applications should never attempt to access or edit Library description files. Instead, applications should always use the Shell programming model or the IShellLibrary API to consume and manipulate Library contents.

In the next post we will dive into the Shell programming model.


Comments

  1. Posted on: April 14, 2009 at 4:06PM  

    As much as I have liked the Windows 7 beta, this one feature that I do not support. It seems to blur the filesystem in unnecessary ways. See my comment from "Understanding Windows 7 Libraries". As I expected, Libraries did add another layer of complexity not found in any previous version of windows. If I want to make a simple program such as "ls" or "dir" I now have to read an XML(indirectly) to make it fully match the functionality of windows explorer.

    I can't wait to get a call from my mom when some of her pictures are missing because she unplugged an external harddrive that was part of her Pictures library. (Side note: What happens if the drive letter changes?)

  2. Posted on: April 14, 2009 at 6:05PM  

    I'm afraid I do not fully understand the point of this feature.  What can a Windows library do that a symbolic link can't?

    As an example, can I not just create a symbolic link pointing to a public folder from the users My Documents folder?  Would this not resolve issues like requiring new software that is 'Windows library' aware etc?  

  3. Posted on: April 16, 2009 at 5:50AM  

    "Users can’t change the default libraries icon via Windows Explorer, nor can they assign the icon to a new customized user-defined library." Why can't this minor behavior change by RTM? Can you allow the icon of any library to be changed/customized by the user from Windows Explorer instead of editing the XML file programmatically?

  4. Posted on: April 16, 2009 at 3:33PM  

    conversely, I'm a huge library fan and what I cannot yet find is how to redirect the Explorer Winkey-E shortcut back to libraries instead of my computer; can this be done in the registry or through GPO?

    Also, the XML suggests libraries can be deployed programmatically/through scripts; what happens if an XML file points at a library that the user cannot add manually (ie one that gives the Cannot add library because it cannot be indexed error?)

  5. Posted on: April 16, 2009 at 4:33PM  

    @MaryBranscombe

    One option is that you can press Win+2 to launch Libraries since it is the second option in the taskbar.

    You make a good point about supported locations. Even though users can not directly add unindexable locations to libraries developers can use the IShellLibrary API to add these add these folders to library. This has the consequence of putting the library in "safe mode" with reduced functionality. Look for more detail on this in Yochay's later posts.

    I want to reiterate Yochay's comment that users and developers should never edit the library description XML directly.

  6. Posted on: September 01, 2009 at 12:54AM  

    with images and code you provided when you were trying to do something but could not manage the information we receive from you and then we will do thanks

  7. Posted on: September 03, 2009 at 4:25PM  

    Hello Yochay,

    How can I know if an item (let's say a file) inside a library was added, removed or changed?

    Thank you,

    Javier Andrés Cáceres Alvis

    Blog Personal: speechflow.spaces.live.com

    Blog Intel: software.intel.com/.../javierandrescaceres

  8. Posted on: January 06, 2010 at 7:19AM  

    You wrote very interesting post i love your blog it contains very informative article i will share it with my friends.

  9. Posted on: January 08, 2010 at 2:17AM  

    Great stuff. I really like your writing style.

Trackbacks

  1. Posted by: Libraries Under the Hood|Join Our Story!|AngNetwork Blog on April 14, 2009 at 2:31PM

    Pingback from  Libraries Under the Hood|Join Our Story!|AngNetwork Blog

  2. Posted by: Links for April 14, 2009 « Steve Mullen’s Blog on April 14, 2009 at 3:35PM

    Pingback from  Links for April 14, 2009 « Steve Mullen’s Blog

  3. Posted by: linkfeedr » Blog Archive » Libraries Under the Hood - RSS Indexer (beta) on April 14, 2009 at 4:08PM

    Pingback from  linkfeedr » Blog Archive » Libraries Under the Hood - RSS Indexer (beta)

  4. Posted by: Tech News, Resources from Blogosphere - 15 April 09(8) | Best Webhosting on April 14, 2009 at 10:42PM

    Pingback from  Tech News, Resources from Blogosphere - 15 April 09(8) | Best Webhosting

  5. Posted by: Windows 7 Libraries Under the Hood on April 15, 2009 at 3:47AM

    Pingback from  Windows 7 Libraries Under the Hood

  6. Posted by: Dew Drop - April 15, 2009 | Alvin Ashcraft's Morning Dew on April 15, 2009 at 8:11AM

    Pingback from  Dew Drop - April 15, 2009 | Alvin Ashcraft's Morning Dew

  7. Posted by: linkfeedr » Blog Archive » Light Up with Windows 7 Libraries - RSS Indexer (beta) on April 16, 2009 at 6:07PM

    Pingback from  linkfeedr  » Blog Archive   » Light Up with Windows 7 Libraries - RSS Indexer (beta)

  8. Posted by: Light Up with Windows 7 Libraries | Windows Seven 7 on April 16, 2009 at 6:12PM

    Pingback from  Light Up with Windows 7 Libraries | Windows Seven 7

  9. Posted by: Light Up with Windows 7 Libraries | Windows 7 Information, News, Downloads, Support Forums on April 17, 2009 at 6:39AM

    Pingback from  Light Up with Windows 7 Libraries | Windows 7 Information, News, Downloads, Support Forums

  10. Posted by: Light Up with Windows 7 Libraries | yoursoftwarelink.com on April 17, 2009 at 8:36PM

    Pingback from  Light Up with Windows 7 Libraries | yoursoftwarelink.com

  11. Posted by: Technology Info » Light Up with Windows 7 Libraries on April 18, 2009 at 7:03AM

    Pingback from  Technology Info  » Light Up with Windows 7 Libraries

  12. Posted by: Windows 7 Libraries Truely Enhance Your Experience | Windows 7 Resource Center on April 18, 2009 at 2:18PM

    Pingback from  Windows 7 Libraries Truely Enhance Your Experience | Windows 7 Resource Center

  13. Posted by: RSS Weekly Digest - #3 on April 22, 2009 at 5:35AM

    Pingback from  RSS Weekly Digest - #3

  14. Posted by: Windows 7 for Developers on April 23, 2009 at 6:16PM

    This is the fifth post about Windows 7 Libraries. By now, you should be familiar with Windows 7 Libraries

  15. Posted by: Consuming the Contents of Windows 7 Libraries | Windows Seven 7 on April 23, 2009 at 7:08PM

    Pingback from  Consuming the Contents of Windows 7 Libraries | Windows Seven 7

  16. Posted by: Consuming the Contents of Windows 7 Libraries | Windows 7 Information, News, Downloads, Support Forums on April 24, 2009 at 12:08AM

    Pingback from  Consuming the Contents of Windows 7 Libraries | Windows 7 Information, News, Downloads, Support Forums

  17. Posted by: linkfeedr » Blog Archive » Consuming the Contents of Windows 7 Libraries - RSS Indexer (beta) on April 24, 2009 at 7:11PM

    Pingback from  linkfeedr » Blog Archive » Consuming the Contents of Windows 7 Libraries - RSS Indexer (beta)

  18. Posted by: US ISV Developer Evangelism Team on April 25, 2009 at 6:27PM

    The release candidate (RC) for Windows 7 is just around the corner. Dates for the Windows 7 RC were announced

  19. Posted by: You Get The . Info » Consuming the Contents of Windows 7 Libraries - 109th Edition on April 27, 2009 at 7:24PM

    Pingback from  You Get The . Info » Consuming the Contents of Windows 7 Libraries - 109th Edition

  20. Posted by: Windows 7 for Developers on May 08, 2009 at 3:20PM

    After two weeks break, we are back to our normal technical posts, covering Windows 7 Libraries. This

  21. Posted by: Windows 7 Libraries ??? IShellFolder | Windows Seven 7 on May 08, 2009 at 4:03PM

    Pingback from  Windows 7 Libraries ??? IShellFolder | Windows Seven 7

  22. Posted by: Windows 7 Libraries ??? IShellFolder | Windows 7 Information, News, Downloads, Support Forums on May 09, 2009 at 9:09AM

    Pingback from  Windows 7 Libraries ??? IShellFolder | Windows 7 Information, News, Downloads, Support Forums

  23. Posted by: You Get The . Info » Windows 7 Libraries Helpers - 165th Edition on May 11, 2009 at 8:16PM

    Pingback from  You Get The . Info » Windows 7 Libraries Helpers - 165th Edition

  24. Posted by: Windows 7 for Developers on May 15, 2009 at 12:30AM

    Here is another post in the Windows 7 Libraries post series. So far, we have covered what Windows 7 Libraries

  25. Posted by: Stay in Sync with Windows 7 Libraries | Windows Seven 7 on May 15, 2009 at 12:53AM

    Pingback from  Stay in Sync with Windows 7 Libraries | Windows Seven 7

  26. Posted by: Stay in Sync with Windows 7 Libraries | Windows 7 Information, News, Downloads, Support Forums on May 15, 2009 at 3:37AM

    Pingback from  Stay in Sync with Windows 7 Libraries | Windows 7 Information, News, Downloads, Support Forums

  27. Posted by: Windows 7 for Developers on May 18, 2009 at 11:52PM

    We have enabled an easy direct access to Windows 7 (and some Windows Vista) features for .NET developers

  28. Posted by: Windows 7 Managed Code APIs | Windows Seven 7 on May 19, 2009 at 12:08AM

    Pingback from  Windows 7 Managed Code APIs | Windows Seven 7

  29. Posted by: Windows 7 Managed Code APIs | Windows 7 Information, News, Downloads, Support Forums on May 19, 2009 at 10:51PM

    Pingback from  Windows 7 Managed Code APIs | Windows 7 Information, News, Downloads, Support Forums

  30. Posted by: Windows 7 Managed Code APIs | Blog on May 20, 2009 at 10:50PM

    Pingback from  Windows 7 Managed Code APIs | Blog

  31. Posted by: Windows 7 for Developers on June 09, 2009 at 4:53PM

    We are wrapping up our comprehensive drilldown into the Windows 7 Libraries API and architecture with

  32. Posted by: Windows 7 Libraries ??? Managed Code | Windows Seven 7 on June 09, 2009 at 5:56PM

    Pingback from  Windows 7 Libraries ??? Managed Code | Windows Seven 7

  33. Posted by: Windows 7 Libraries ??? Managed Code | Windows 7 Information, News, Downloads, Support Forums on June 10, 2009 at 2:24AM

    Pingback from  Windows 7 Libraries ??? Managed Code | Windows 7 Information, News, Downloads, Support Forums

  34. Posted by: Windows 7 for Developers on June 11, 2009 at 11:51AM

    Last updated: June 11, 2009 This post is an index to the recent Windows 7 Libraries series. It includes

  35. Posted by: Windows 7 Programming Guide ??? Libraries | Windows Seven 7 on June 11, 2009 at 12:23PM

    Pingback from  Windows 7 Programming Guide ??? Libraries | Windows Seven 7

  36. Posted by: BloggersBase Computers on June 11, 2009 at 1:42PM

    Windows7 have both positive and negative features.All cannot be found at a one place there are some reference about the program guide and library features which i go through the msn blog while...

  37. Posted by: BloggersBase Computers on June 11, 2009 at 1:52PM

    Windows7 have both positive and negative features.All cannot be found at a one place there are some reference about the program guide and library features which i go through the msn blog while...

  38. Posted by: Windows 7 Programming Guide ??? Libraries | Windows 7 Information, News, Downloads, Support Forums on June 12, 2009 at 12:34AM

    Pingback from  Windows 7 Programming Guide ??? Libraries | Windows 7 Information, News, Downloads, Support Forums

  39. Posted by: You Get The . Info » Windows 7 Programming Guide ??? Libraries - 300th Edition on June 16, 2009 at 9:32PM

    Pingback from  You Get The . Info » Windows 7 Programming Guide ??? Libraries - 300th Edition

  40. Posted by: Windows 7 Programming Guide – Libraries « Windows Club on June 20, 2009 at 1:20AM

    Pingback from  Windows 7 Programming Guide – Libraries « Windows Club