In search of better tools for NTFS reparse points (junctions, symlinks)

Back in 2006 when solid state drives were fairly new and their sizes were pretty small I found myself with a problem. The drive was too small to hold both the operating system, my programs and all of my documents, photos, videos, etc. While it was acceptable to simply save all of my files to a second, larger, hard drive it wasn’t exactly intuitive the way working with the Documents and Settings or Users folder was.

When I first discovered Junctions I thought I had found the perfect solution. With a junction I could copy the entire C:\Users directory to a totally different drive and then trick Windows into thinking it was still on my C: drive with a junction. After some concerted effort I did just that. The problem with my approach though is that it requires manually recreating all of the junctions and symlinks hidden throughout the C:\Users directory tree. Ouch!

What I have needed for so long was some good tools to make the process easier (and less error prone). Even as of 2014, robocopy still only has bare bones support for symlinks while junctions are entirely broken. So today I am releasing my efforts to dramatically expand the tools available for working with NTFS reparse points on Windows. It’s called ntfslinkutils.

TL;DR

ntfslinkutils is a collection of command line utilities for Windows designed to copy (cplink), move (mvlink), delete (rmlink) and even fix (fixlink) junctions and symbolic links. With these tools you can manipulate every junction and symlink in an entire directory tree and even rewrite the targets of each along the way.

As per usual I am releasing the code as open source under the generous BSD license. You can find pre-built binaries available for download here. Note you may need to install the Visual C++ Redistributable for Visual Studio 2012.

I hope you find these new tools as useful as I do!

Update 2014/9/18: I have removed the Visual C Runtime Library requirement in order for the tools to run under the Windows Pre Installation Environment (PIE). While I have tested this with Windows 8.1 I am unsure if they work with previous versions. Please let me know in the comments if you have a chance to try them on Windows Vista/7.

Working with NTFS reparse points in C++

Probably one of the lesser known features of NTFS are reparse points. A reparse point is a virtual path on the file system that points to some other place. If you’re familiar with *nix systems this sounds an awful lot like a symbolic link which is in essence exactly the same thing. Reparse points come in two flavors; junctions and symbolic links. Both allow you to point to some arbitrary path. So what is the difference?

Junctions were one of the first types of reparse points built in to NTFS. They first saw the light of day in Windows 2000. A junction acts much more like a hard link within the file system with the exception that deleting one will not delete the underlying contents. Their limitation is that they can only point to paths on the same or different local volumes on the machine. You cannot for instance make a junction to a network share nor can you make a junction to a file.

Symbolic links on the other hand remove the limitations of junctions and offer the ability to target virtually any arbitrary path. This effectively makes them junctions 2.0 and on par with the *nix counterpart. However, symlinks can only be created with administrative privileges and some parts of the operating system treat them differently whereas junctions are often treated as if they are still a hard link.

Unfortunately Microsoft have made it pretty difficult to work with both. The Windows SDK provides a rather arcane API in the form of DeviceIoControl to work with junctions and the API has a couple of key missing features when working with symbolic links. In addition, the only other available library I could find was severely outdated and doesn’t handle symlinks (see Windows 2000 Junction Points).

So I decided to write a new library. libntfslinks is a modern C++ library for Windows 7/8 that simplifies working with NTFS junctions and symbolic links. The library’s features are:

  • Create junctions and symbolic links
  • Delete junctions and symbolic links
  • Extract the target path of junctions and symbolic links
  • Determine if a path is a junction or a symbolic link
  • Unicode support
  • 32-bit and 64-bit compatible

The library’s API is written to be as simple as possible without any fluff. To create a junction simply call CreateJunction, to check if a path is a symlink use IsSymlink. That’s it!

The code is open source under the BSD license and available on github along with instructions on how to build and use the library.

https://github.com/caskater4/libntfslinks

Moving your data where you want in Windows Vista/7

Microsoft has graciously decided to get rid of the Documents and Settings folder in favor of the new Users folder in Windows Vista/7. This is nice because not only is it faster to type but its a little more familiar for Unix/Linux users. The one thing I still would love to see in the Windows installer, however, is the ability to choose the location of this Users directory (and heck even rename it). Unfortunately MS does not provide us that ability by default. To get that ability you have to modify the actual Windows installer.

So what if you don’t want to have to re-install the operating system? Maybe you just want to move your data to the place of your choosing without any serious modification to your system. There are a couple solutions out there people have already written about.
They are:

  1. Change the location of each data folder (i..e Documents, Pictures, Music) individually using Windows explorer settings.
  2. Modify the Windows registry to reflect the new folder location

There are a couple of problems with these two solutions. The first solution means that you can only change specific folders such as Documents, Pictures, Music, etc. It doesn’t solve the problem of moving hidden application data or other folders to the new location. It also becomes tedious to do so. Finally, if you want to move all of your user profiles to a new location each user must move the data their self. All in all its not really a solution to the problem.

While the second solution will work it is very dangerous to edit the Windows registry. Making even minor changes to the registry can render your entire system useless. I actually tried this method on Windows XP and was unable to get it working. The problem with editing the registry is that there are hundreds if not thousands of references to the profile folders within it and you may not find every one. Missing even one will prevent Windows or your software from working properly or even at all.

Thankfully I found an easier way.

This method requires that you use the Windows installation disc or use a WinPE 2.0 boot disc. To learn how to create a WinPE 2.0 disc go to http://www.msfn.org/board/lofiversion/index.php/t83722.html. Regardless of your choice you will need to get to the command prompt. If you are using the WinPE disc you should be immediately given a command prompt once it has finished booting. If you are using the Windows installation disc you can find instructions how on how to get to the command prompt at http://www.bleepingcomputer.com/tutorials/tutorial147.html.

WARNING: The following can be dangerous to your system. As with any hack you should always fully back up your entire hard drive just in case anything goes horribly wrong. If something bad happens to your system don’t blame me… I warned you ;)

For this example I am assuming that you want to relocate your Users directory to your D:\ drive. If you want to place it elsewhere, just change the commands accordingly. Now that you are at the command prompt, type the following commands followed by enter for each:

  1. xcopy /E /H /O /X /Y /I C:\Users D:\Users
  2. rmdir /S /Q C:\Users
  3. rmdir “C:\Documents and Settings”
  4. mklink /J C:\Users D:\Users
  5. mklink /J “C:\Documents and Settings” D:\Users

What you’ve done here is copied everything in the Users directory to the new location and then created an NTFS junction to the new location. You might also notice that Documents and Settings showed up again. You thought Vista got rid of it for good didn’t you? Many applications are hard coded to use the Documents and Settings directory and do not know that Vista has changed this. To make sure these applications still work under Vista Microsoft cleverly created their own junction from the older Documents and Settings to their new Users. You might ask why is it necessary to do change the existing junction then. Well as I discovered myself NTFS for some reason has a bit of trouble with accessing a junction of a junction. So when I first attempted this hack I encountered problems.

You may now restart your computer and you should have successfully moved your entire Users directory to your new location. Congratulations!

Update: After some further testing i’ve discovered something interesting. When using xcopy to move the data files to a new location the special folder icons for Documents, Pictures, Searches, Music, etc. appear to be lost. Also, performance when booting up again may seem slightly slower at first. So instead of using xcopy I tried using a backup program like TruImage to backup the Users directory and then restore it in the location I desired. Using this kept all the special folder icons and the system appeared to be a bit more snappy at first bootup. So while the method described works just fine, for performance and aesthetic reasons I would say backup/restore your Users folder instead. Thus the steps would be as such:

  1. Backup C:\Users using a backup program
  2. Restore the C:\Users to your new desired location (example: D:\Users)
  3. Start WinPE or Windows Installation disc command prompt
  4. Using the command prompt
    1. rmdir /S /Q C:\Users
    2. rmdir “C:\Documents and Settings”
    3. mklink /J C:\Users D:\Users
    4. mklink /J “C:\Documents and Settings” D:\Users
  5. Restart

Update (Oct 02, 2012): After spending some time working on a new install recently I discovered that robocopy is a perfectly suitable replacement for both xcopy and a backup program. In my testing I found the following command options work perfectly to create an accurate copy.

robocopy /B /COPYALL /MIR /XJ <Source> <Destination>

It works so good in fact that I use the command internally in my new MoveUserData utility program I wrote to automate the whole process. I have detailed the new program in an article here.