Data Hoarding: Building a NAS

In October 2023, I received a message from Microsoft stating that all screenshots and game recordings on the Xbox Network would be deleted after 90 days. This included all existing media on Xbox’s cloud, which previously had no restriction on retention.

This change didn’t come as a major shock to me. As cloud services continue to mature, it seems predictable that policies will adapt. Ultimately, this was an overdue wake-up call to actively backup and save my files from being lost. After all, I had nearly a decade of data taken across multiple generations of consoles; data I trusted would always be readily available to me on the cloud.

This updated policy would take effect at the beginning of 2024, giving me enough time to comb through my captures, determine what was worth keeping, and store it elsewhere (in my case, Google Drive). Great, crisis averted.

While reviewing my old videos and screenshots (Destiny, Halo), nostalgia swayed me to not only maintain these memories, but to get back into the habit of doing more recording. Over the following week, I began a thorough search of any remaining gameplay footage that I may still have that wasn’t uploaded to the Xbox Network. The most notable finds came from booting up my old Xbox 360, of which the hard drive contained hours of Halo: Reach footage that I thought was lost.

It was a great feeling to centralise my console data and to reignite a long-forgotten hobby. However, two major concerns arose:

  1. My standard 15GB Google Drive limit would not be nearly enough storage moving forward.
  2. What if Google implements a similar retention policy? Where do I put my data then?

The answer to both of these questions – and ultimately my excuse to learn about some new tech – was to build my own NAS, or network attached storage.

While the simplicity of using SaaS cloud storage like Google Drive or OneDrive had been satisfactory for many years, Microsoft’s changes to the Xbox Network left me with a desire to try my hand at taking control of my information. In doing so, I wouldn’t need to worry about corporate policies changing that would put my data at risk, and putting together an adequately sized storage pool means I don’t need to be concerned with running out of space.

The Build

Fast forward to today, my NAS has already been built and deployed. In the spirit of being concise, there are only three characteristics about the build worth noting: hard drives, operating system, and redundancy level.

For my drives, I chose two 4TB Western Digital Red Plus drives. These HDDs are rated for 24/7 usage in NAS environments, making them a solid choice over regular consumer drives. They also benefit from the performance of conventional magnetic recording (CMR) technology, as opposed to shingled magnetic recording (SMR), an approach that involves overlapping the magnetic tracks data is written to. SMR drives allow for higher storage capacity at the cost of hindered read and write speeds. 

For the OS, TrueNAS SCALE was the winner. This is a Linux-based solution designed to deploy storage pools with the ZFS file system. The primary feature that drew me to ZFS was the calculation of checksums on stored data, allowing the system to report on any potential corruption. Since my use case is largely archival and many of my files won’t be accessed regularly, this is nice to have. In addition to this, TrueNAS’ web UI allows for automating and reporting these checksum health checks, which significantly reduces the manual setup required.

Perhaps the most important consideration when building any sort of storage system is the redundancy level. Because I am only working with two physical drives, a mirrored storage pool was the obvious choice. Mirrored storage means data is copied equally between both disks. Because of this, I lose half of my overall capacity (giving me 4TB of usable space from the 8TB total), but in the event that one of these drives were to die, my data would still be accessible. This solution provides me with a level of fault tolerance I’m comfortable with, with the option for expansion in the future.

Where To Go From Here

I’ve always been diligent with what I choose to hold on to, so I don’t expect to fill my storage pool for a long time. The most common term I see online to describe NAS and data enthusiasts is “data hoarder”. I think “preserver” is probably more fitting in my case. Okay, maybe “paranoiac” fits to a degree, too.

The truth is, there is so much information that is no longer accessible or becomes lost altogether. Just last month, the Xbox 360 Store was shut down. Like me, you may have initially thought to yourself, “that was still running?”. Yes, it was, and it hosted numerous games that were exclusively available to the 360 Store with no physical or otherwise official means of purchasing. There are likely unofficial emulation options that exist, but it’s terribly sad to think that many titles can no longer be officially purchased.

Cases like the Xbox Cloud or the 360 Store make you realise how impermanent data can be.

Whether I cover this topic again in the future or not, this was a really interesting project and learning opportunity. Beneath all of the tech, however, the motivating factor was to maintain a lifetime of memories I’ve made with friends and family over games.

With my immediate problem solved and room to grow, my use case will likely expand beyond the scope of video games. For now, it’s reassuring to know I’ve taken steps to protect this data. We’ll see how deep this rabbit hole goes as I continue to learn.