In search of better tools for NTFS reparse points (junctions, symlinks)

Back in 2006 when solid state drives were fairly new and their sizes were pretty small I found myself with a problem. The drive was too small to hold both the operating system, my programs and all of my documents, photos, videos, etc. While it was acceptable to simply save all of my files to a second, larger, hard drive it wasn’t exactly intuitive the way working with the Documents and Settings or Users folder was.

When I first discovered Junctions I thought I had found the perfect solution. With a junction I could copy the entire C:\Users directory to a totally different drive and then trick Windows into thinking it was still on my C: drive with a junction. After some concerted effort I did just that. The problem with my approach though is that it requires manually recreating all of the junctions and symlinks hidden throughout the C:\Users directory tree. Ouch!

What I have needed for so long was some good tools to make the process easier (and less error prone). Even as of 2014, robocopy still only has bare bones support for symlinks while junctions are entirely broken. So today I am releasing my efforts to dramatically expand the tools available for working with NTFS reparse points on Windows. It’s called ntfslinkutils.

TL;DR

ntfslinkutils is a collection of command line utilities for Windows designed to copy (cplink), move (mvlink), delete (rmlink) and even fix (fixlink) junctions and symbolic links. With these tools you can manipulate every junction and symlink in an entire directory tree and even rewrite the targets of each along the way.

As per usual I am releasing the code as open source under the generous BSD license. You can find pre-built binaries available for download here. Note you may need to install the Visual C++ Redistributable for Visual Studio 2012.

I hope you find these new tools as useful as I do!

Update 2014/9/18: I have removed the Visual C Runtime Library requirement in order for the tools to run under the Windows Pre Installation Environment (PIE). While I have tested this with Windows 8.1 I am unsure if they work with previous versions. Please let me know in the comments if you have a chance to try them on Windows Vista/7.

Working with NTFS reparse points in C++

Probably one of the lesser known features of NTFS are reparse points. A reparse point is a virtual path on the file system that points to some other place. If you’re familiar with *nix systems this sounds an awful lot like a symbolic link which is in essence exactly the same thing. Reparse points come in two flavors; junctions and symbolic links. Both allow you to point to some arbitrary path. So what is the difference?

Junctions were one of the first types of reparse points built in to NTFS. They first saw the light of day in Windows 2000. A junction acts much more like a hard link within the file system with the exception that deleting one will not delete the underlying contents. Their limitation is that they can only point to paths on the same or different local volumes on the machine. You cannot for instance make a junction to a network share nor can you make a junction to a file.

Symbolic links on the other hand remove the limitations of junctions and offer the ability to target virtually any arbitrary path. This effectively makes them junctions 2.0 and on par with the *nix counterpart. However, symlinks can only be created with administrative privileges and some parts of the operating system treat them differently whereas junctions are often treated as if they are still a hard link.

Unfortunately Microsoft have made it pretty difficult to work with both. The Windows SDK provides a rather arcane API in the form of DeviceIoControl to work with junctions and the API has a couple of key missing features when working with symbolic links. In addition, the only other available library I could find was severely outdated and doesn’t handle symlinks (see Windows 2000 Junction Points).

So I decided to write a new library. libntfslinks is a modern C++ library for Windows 7/8 that simplifies working with NTFS junctions and symbolic links. The library’s features are:

  • Create junctions and symbolic links
  • Delete junctions and symbolic links
  • Extract the target path of junctions and symbolic links
  • Determine if a path is a junction or a symbolic link
  • Unicode support
  • 32-bit and 64-bit compatible

The library’s API is written to be as simple as possible without any fluff. To create a junction simply call CreateJunction, to check if a path is a symlink use IsSymlink. That’s it!

The code is open source under the BSD license and available on github along with instructions on how to build and use the library.

https://github.com/caskater4/libntfslinks

Relocating Windows User Data

Quite some time ago I wrote a post about how to relocate the C:\Users and C:\ProgramData directories of a Windows installation to another drive. This is useful for those of us whom want to use a different hard drive for the operating system and keep documents elsewhere. In my case, my OS is installed to a solid state drive that is simply too small to fit all of my documents, music and pictures. I simply needed a better way to store all the data. When I discovered NTFS junctions (aka: reparse points) I realized I could do just that. I could finally put some breathing room between my operating system and my data.

Now that Windows 8 is here (well almost) I realized I was going to need to move all that data over yet again. In previous years, relocating user data was a pain, specifically because of all the reparse points that have to be fixed recreated once the data is copied to the new drive. Making sure you recreate every reparse point can be quite challenging. Moreso, if you miss one it could mean very bad things when you try to boot the system up again. This got me thinking, why not just write a program to do all that dirty work for me? And so I did.

The program is called MoveUserData. It runs at a command prompt and is intended to be used in the Recovery Console (so that security rights are not an issue). Below you will find the program and full source code.

MoveUserData_v1.zip 
MoveUserData_source.zip

Using the program is quite simple. Copy MoveUserData.exe to a USB drive or CD. Boot the Windows installation disc and go into Windows Recovery mode. Choose Command Prompt from the list of recovery options. Once you are at a command prompt put the CD/USB drive containing MoveUserData.exe into the computer.

Assuming that your Windows installation drive (normally C:) is currently listed as K:, your data drive is D: and that the CD/USB drive is set to E: then you can execute the following commands.

The program will do the rest. Once done, just reboot!

Disclaimer: As stated in my previous article, messing around with your operating system like this is very dangerous. While I have made every effort to ensure the program works as described it is still possible that errors will occur. So make sure to back up your drive first. Use of this program is at your own risk.

Update [October 5, 2012]:
So despite my rigorous testing of the program it appears that it doesn’t actually work in the scenario in which I intended, the command prompt of the Windows Preinstall Environment (installer discs). Truth be told I never actually tested it in this scenario because it would have made development incredibly slow and complicated. Instead I tested using a duplicated data set on a separate pair of drives from my main system. Apparently that was a big mistake on my part since it would seem .NET based applications like mine don’t run at all in Windows PE. Bummer!

An aside: Why on earth would Microsoft develop the .NET framework and market it as the end-all-be-all of Windows’ platform development if they don’t include support in Windows PE? That seems like a huge gap in judgement.

Back to the real issue at hand. Since no C#/.NET programs will work I am rewriting the utility entirely in C++. So stay tuned!

Dissecting the Xyloband

A couple of nights ago I went to the Coldplay concert at the Hollywood Bowl. The show was amazing in every respect but what really grabbed my attention were the wristbands they handed out at the gate. These wristbands immediately lit up and began to flicker to the beat of the music, and coordinated with the rest of the lighting coming from the stage (to see it in action take a look at this video). It was truly impressive to say the least!

Being naturally curious as I am I started researching the little gizmo as soon as I got home. I found this great post on Hackaday that already began the discussion I was interested in; how does this thing work? That quickly led me to the interview with it’s creator, Jason Regler, and finally to the actual patent filed with the USPTO (of which I proceeded to read in detail).

My next step was to actually take the thing apart. I’m lucky to have two of these since my fiance Heather joined me on the occasion.

You can see from the picture above that it is powered by three 3V batteries. The two CR2016 batteries are stacked on the back side of the board while the CR2032 is attached to the front. The circuitry itself seems moderately complex with only two ICs and a number of resistors. The most visible IC is on the bottom left is an Si4313 RF receiver from Silicon Labs. This RF receiver runs in the 240-960 Mhz range. The second IC that is partially visible from underneath the battery clamp appears to be the C8051F980 Microcontroller (MCU), also from Silicon Labs.

And for good measure here is a shot of the back of the circuit board.

It’s a very cool gadget. As I dig a little deeper I hope to be able to turn this thing on at my own will, preferably without Charlie Brown’s help.

 

Multi-Touch Table – Construction

After having the opportunity to play with Microsoft Surface first hand I immediately wanted to have one. However after I discovered the hefty price tag for one these tables and the fact that they are not yet available to consumers I decided to build my own. With some help from the great people at NUI Group I am well on my way to completing my new multi-touch coffee table for my living room. The table has been constructed and I am in the process of putting all the final parts together. Stay tuned for updates!

The main construction of the table is complete. I’ve constructed the box out of plywood and use 2×2’s for structural support on the sides. No nails were used around the frame of the table (just a lot of awesome glue).

The big empty space you see on the side is where the access door goes.

 

ventman

If you’re running one or more Ventrilo Servers on a Unix/Linux machine you’ve undoubtedly noticed the lack of autorun scripts and other tools to keep track of those servers. Ventman was created to fill that void. It is an easy to use command line utility for managing multiple Ventrilo servers. If you need to restart all your servers its as easy as ventman restart all.

ventman is released under the GNU General Public License version 3.

Downloads

ventman-1.0.zip