- Update 2023-09-25: tagging modifies file hashes
This quite lengthy article explains and discusses the built-in file tagging implementation of Microsoft Windows 10. I do have a strong background with PIM and tagging and this article is written from the human perspective when manually tagging user-generated files. Please do read my general recommendations on using tags in an efficient way.
To my knowledge, Microsoft is currently not actively promoting this feature. Therefore, complaining on bad design decisions does not apply here as long as Microsoft does not understand this kind of tagging as something which was designed to be used by the general user. Because from my perspective, it obviously can't be meant to be used in practice. Unfortunately. Let's take a closer look why I came to this conclusion.
TL;DR: Microsoft Windows does provide NTFS features to tag arbitrary files. Some applications do also merge format-specific tags with these NTFS tags. Although there are quite nice retrieval functions for tags, it is very complicated to use this for general file management. Applied tags are easily lost so that in practice, users will refrain from using native Windows file tagging like this.
Table of contents:
- What Does Tagging Mean Here?
- A Well-Hidden Feature
- How to See and Assign Tags
- How to Make Use of Tags
- Playing Around With Tags
- Enabled File Types for Tagging
- How to Enable Tags for More File Types
- Relations Between Applications and Meta-Data
- History, Implementation Details, and Similar Implementations
- Windows 10 Tags Considered as Fragile
- Summary and Remarks
What Does Tagging Mean Here?
For this article, I am talking about non-collaborative local file-tagging. This describes the process of attaching one or more unique keywords to files stored on NTFS file systems by users who are able to access the file with granted write-permissions via the Windows File Explorer. "Keywords" and "tags" are used as synonyms here.
I could elaborate on tag and tag-system definitions for quite some time but let us stop here for the sake of brevity. It will be a long journey after all.
A Well-Hidden Feature
By default, the Windows UI does not expose anything at all that would help the users to recognize the file tagging possibility. So we do have a more or less full support for tagging files and yet Microsoft hides this quite well from the common eye. Probably for a good reason, which we are going to find out below.
Although I'm very interested in topics related to tagging this feature is that well hidden so that I was not aware of this feature myself until I read about it in a book in 2018. Support for tagging started as early as with Windows Vista.
How to See and Assign Tags
In order to see and edit file tags, you have to enable "View (Tab) → Details pane" in the File Explorer.
There is a second UI feature you might want to activate: the read-only Tags column is activated by choosing "Tags" in the context menu of the column bar:
When you go through different files, you will recognize that not all file types can be tagged by default. For example, the details pane for a simple text file does not show the "Tags: Add a tag" in contrast to any JPEG image file as shown in the screen-shots above.
Assigned tags are visible in the details pane as well as in the tags column:
Adding or modifying tags is possible in the Details pane but not in the tags column. You will recognize that Microsoft allows tags with spaces and special characters. Multiple tags are usually separated by semicolons which is probably the only standard character which is not allowed within tags.
The last place where File Explorer is showing you the assigned tags and also allows to edit them is within the Properties of a file:
As shown in the screenshots above, tags might be added/removed/modified at two places: either on the "Details pane" (on the right hand side of the File Explorer window) or within the file properties on its "Details" tab.
How to Make Use of Tags
Now that we have tagged some files, what possibilities are there to use this meta-data in daily life? First of all, there is navigation. For navigating through your files, you might prefer your File Explorer sorted alphabetically by file name:
With tags, you might also sort alphabetically by tags instead:
Since the order of files in the "sorted by tags"-view is depending on the order of tags within the files, I do not consider this a great improvement. However, what is really neat is when you consider the "Group by"-method. Be default, File Explorer is grouping by names:
You can change the grouping in the "View" tab of the File Explorer:
Having switched to "Group by Tags", you will notice that all files are arranged by their assigned tags:
Untagged files are listed in the "Unspecified" category at the bottom. The categories above correspond to the alphabetically sorted list of tags. Each file is listed once for each tag. So if a file like JPEG file 3.jpg
does have two different tags ("Dogs" and "House"), it is listed twice. One time in the category "Dogs" and one time in the category "House". If you select it in one category, this single file gets selected in all categories.
Complementary to file navigation, File Explorer has a search feature implemented. The following image shows the result when you do search for a tag "house" within the folder we've used above:
You will notice that all files are listed in the results that do feature the tag "house" or "House". So search as well as "Group by Tags" is case insensitive when it comes to tags. All other files, not having the "house" tag, are omitted.
When you search for multiple tags, just the files that do contain all of them are listed:
On the negative side, you can not search for keywords that only occur within tags. I would have expected a query language according to the widespread pattern like "tag:dog" which would look for the occurrence of "dog" but only within the tags and not the file name or the content.
So if you're searching for "dog", you will find files that contain the tag dog as well as files that do contain "dog" within their file name:
This File Explorer tag search is not a sub-string search: if you want to find files tagged with "mydog", you can not find them by searching for "dog". However, when you have tagged files with "my dog", you will find them in the search results for "dog" but not within search results for "dogs".
In summary: Searching for tags is:
- case-insensitive,
- non-sub-string,
- whole-word and not whole-tag.
Playing Around With Tags
When you play around with different tags, you will find out that this feature is intended to be used case-insensitive. When you tag a file with "Dog" and "dog", the last one wins and the other gets removed.
When "Arrange by Tags" is used, the tag "Dog" as well as "dog" gets listed in the category "Dog".
When you select multiple tagged files, the Details pane shows only the tags that can be found within all selected files. The other ones are not visualized. You may add additional tags which then gets added to all selected files:
You may remove all tags of one or a set of selected files with "Properties → Details → Remove ...".
This page mentions a context menu function to export the meta-data of selected files to an xml
file. Meta-data from an xml
file could be applied to the files as well. I was not able to find this function in my tests.
Enabled File Types for Tagging
In the previous sections I mentioned briefly that only a sub-set of file types may be tagged by default. In my opinion, this is a very tough restriction if you want to use tags for organizing your files.
On a fresh Windows 10 installation, there are not even a hundred file types that may be tagged. When apps get installed like Microsoft Office or LibreOffice, meta-data handlers for additional file formats gets added and configured. On my business Windows 10 system approximately 180 extensions had associated meta-data handlers. After installing LibreOffice on a Windows 10 virtual machine, about 120 extensions were listed as tag-able, approximately thirty of them from LibreOffice alone. I noticed that LibreOffice does not create meta-data handlers for Microsoft formats such as .docx
or .xlsx
whereas handler for older formats are created: .doc
or .xls
.
It is important to know that not all meta-data handlers offer meta-data tagging by keywords. Only meta-data handlers that contain definitions for "System.Keywords" result in the ability to be tagged. Furthermore, not all meta-data handlers that contains keywords/tags offer them also in file properties.
I tried to come up with a minimum list of activated tagging via meta-data handlers. When downloading a fresh Windows 10 virtual machine like that one, you will find some tools pre-installed. In this case, these are many development tools. After manually installing DotNet, LibreOffice 5.4.4, paint.net 4.2.5, all extensions with enabled handlers for keywords/tags are:
.asf .cr2 .crw .dng .doc .dot .dvr-ms .erf .flac .jfif .jpe .jpeg .jpg .jxr .kdc .m1v .m2t .m2ts .m2v .m4a .m4b .m4p .m4v .mka .mkv .mod .mov .mp2 .mp2v .mp4 .mp4v .mp3 .mpeg .mpg .mpv2 .mrw .msi .msp .mts .nef .nrw .pef .raf .raw .rw2 .rwl .sr2 .srw .tif .tiff .tod .ts .tts .uvu .vob .wdp .weba .webm .wma .wmv
I did not mention all well-known LibreOffice formats that were also in the list.
As you can see, most of these activated file types do not reflect bug relevance for the average user. Selected extensions that do not have handlers or no handlers that provide tagging:
.avi .docx .exe .gif .lnk .mp3 .png .wav .css .csv .epub .gz .html .json .java .txt .wmf .xhtml .xlsx .zip
Therefore, there are many file types which may be used on any given Windows machine that can not be tagged by default.
How to Enable Tags for More File Types
After we have found out that it would be nice to have more file formats enabled for tagging, how are we able to enable meta-data handlers ourselves?
The answer lies within a project called FileMeta. You can download the latest release on their release page. Installing this tool requires administration permissions. I totally recommend the documentation pages for learning about details on this topic in general.
After installing FileMeta, you will find multiple executables in its install directory: FileMeta.exe
, FileMetaAssoc.exe
and FileAssociationManager.exe
.
Most things can also be done on the command line. For configuring the tagging functionality, we'll stick to the graphical FileAssociationManager.exe
for this article. After starting up the File Meta Association Manager you will see three main parts of the UI:
- Some workflows for manipulating on the left hand side,
- the File Extensions list with the handler associations and
- the meta-data related settings on the right hand side:
Extending the List of File Extensions
The list of the file extensions are read from the Windows registry. If you can not find a specific file extension in the File Meta Association Manager, no application has registered the file extension so far. If you do associate a file extension with an application ("Always open with ..."), this does not create a registry entry. Therefore, associating an extension with an application is not sufficient that this extension gets listed in the File Meta Association Manager.
To add an extension not listed yet, you have to start the registry editor with administrator privileges, go to "HKEY_LOCAL_MACHINE" → "SOFTWARE" → "Classes" and choose "New → Key" from the context menu.
Then you can enter your new extension like, e.g., .org
and confirm with the return key. After restarting the File Meta Association Manager you'll find the new extension in the list.
Pre-Defined Profiles
My File Meta Association Manager lists two pre-defined profiles: "Simple" and "OfficeDSOfile". The latter seems to be set up by LibreOffice. The "Simple" profile has a few properties set up for "Preview Panel", "Details tab in Properties" and "Info Tip":
Custom Profiles
If you would like to set up a new custom profile, you have to know:
- Full details → Description "System.Keywords": necessary to see and edit tags in the preferences → Details tab.
- The Preview Panel → "System.Keywords": necessary to see and edit tags in the Details pane.
You can't have Details pane without preferences Details tab. Both settings enable the tags shown in the column bar.
Therefore, a minimal custom profile for tagging where you can see the tags in the Details tab looks like that:
Such a profile results in a File Explorer view like that, where you can edit tags in the preferences as well as in the Details tab:
Whenever you change meta-data handlers, you will probably going to restart the File Explorer via the "Restart Explorer" button of the File Meta Association Manager in order to apply changes.
After setting up a custom meta-data handler for file extensions, you can see them also in the command line tool FileMetaAssoc.exe
:
c:\Program Files\File Metadata>FileMetaAssoc.exe -l .txt Simple File Meta Property Handler c:\Program Files\File Metadata>
Relations Between Applications and Meta-Data
As mentioned briefly before, some applications do create meta-data handlers for file extensions when being installed. For example, LibreOffice is creating handlers for their document formats as well as some formats from Microsoft such as .doc
or .xls
but not .docx
or .xlsx
.
Programs like LibreOffice Writer or Microsoft Word do provide meta-data within the preferences of an open document.
You are able to enter tags within the document properties:
These tags can now be seen in the file properties (Details tab) as well as in the tags column. Because of the missing "System.Keywords" in the profile for the "Preview Panel", the tags are not shown in the Details tab of the File Explorer:
Here is the File Meta Association Manager profile "LibreOffice property handler" as set up by LibreOffice:
It's interesting to see that the "LibreOffice property handler" is not visible in the File Meta Association Manager profiles. So I tried to overwrite the "LibreOffice property handler" with the "Simple" profile. To my surprise, this happened:
Yes, this makes sense after all. After confirming this dialogue, the File Meta Association Manager window was gone. I thought that this action was not successful and the app crashed. After restarting the application, I noticed the successfully merged profiles for the .odt
extension.
Unfortunately, in contrast to my expectations, there was no change: no tags visible in Preview page of File Explorer and tags in Details tab can not be changed, only viewed. So this was not a success after all: I still can not modify tags for LibreOffice Writer files outside of LibreOffice Writer file preferences although they can be seen in File Explorer.
So I started to create some non-native LibreOffice Writer documents: .doc
and .docx
. For .docx
files, there were no document property tags visible in File Explorer: not in Preview pane, not in tags column and not in the file properties.
Different story with the .doc
files though: Here, the document property tags are synchronized with the NTFS meta-data. Whenever a tag is added or changed in the file properties, the same change appears in the LibreOffice Writer document properties and vice versa. However, there are no tags/keywords visible in the Preview pane.
This tag synchronization mechanism has a minor issue: when you do not create a .doc
file from within LibreOffice Writer or Microsoft Word but with a text editor, there is no within-file meta-data preferences yet. This results in an error message when you want to tag a zero byte .doc
file in File Explorer:
When you do select "New → Excel Spreadsheet" in File Explorer with Microsoft Office installed, it does not create a zero byte file as with Word files using the same method. Instead, it fills the spreadsheet file with a seven kilobyte default content. This way, you won't get this error message for Excel files in this situation.
Related to this, you can read on the FileMeta FAQ for PDF files:
If I add the File Meta Property Handler for PDF files, will I see properties already in those files? No, unless you are using version 1.4 and are extending an existing property handler for PDF files. File Meta has no capability otherwise for reading properties held within the PDF formatted part of the file. File Meta always writes properties in an NTFS-provided annex to the file. [...] The bad news is that File Meta before version 1.4 will not read properties held in the type-specific formatted part of a file, and no version of File Meta will update such properties.
To make this even more complicated, you have to know that Windows supports tags for every file type, internally. They will not be visible in the properties section of that file, but when you search for those tags, the file appears in search results.
After all these experiences I can only sum up my experience with: it's very complicated. The end-user can not expect tags/keywords to be visible in the File Explorer. She is not able to know if document preference keywords are synchronized to the NTFS meta-data. If there are tags visible, they may not be able to be managed on the Preview pane or the file preferences. File Explorer search seems to find all keywords so far. However, you don't know that a specific file was found because of a tag or anything else since this visualization is missing.
History, Implementation Details, and Similar Implementations
You can read about the history of this feature and some technical details on this page. Basically, NTFS stores the meta-data within an Alternate data streams (ADS). This is quite similar to how Apple stored meta-data in HFS+ and probably also within AFS. I was using the color labels of OS X up to Leopard. They ended up as file-system based meta-data as well.
You can read on this Wikipedia article:
In Apple's macOS, the operating system has allowed users to assign multiple arbitrary tags as extended file attributes to any file or folder ever since OS X 10.9 was released in 2013, and before that time the open-source OpenMeta standard provided similar tagging functionality in macOS.
Windows 10 Tags Considered as Fragile
I do think that the average reader does agree that using tags with this Windows 10 feature is a drag from the user experience point of view already. I do have sad news: this now even gets worse.
Since meta-data are stored in NTFS data streams, you are losing all of the tags when files get moved to someplace where there are no NTFS data streams or when applications generating files do not respect them properly. As a consequence, there are many possibilities where meta-data gets lost. Here is a list of the most obvious ones.
- Losing meta-data when copying to a thumb drive
- Copying a tagged file to a drive that is not formatted with NTFS results in a silent loss of the meta-data. Thumb drives usually are formatted with FAT32.
- Losing meta-data when sending them via email
- When you attach a tagged file to an email, the meta-data does not get attached as well.
- Losing meta-data because of applications handling temporary
files
- When you open a file in too many Windows applications, new modifications by the user get written to a temporary file. On saving the changes to the file, this temporary file then gets renamed to the original file name, overwriting the previous file as well as the meta-data. This is a very mean behavior since users would never expect to lose meta-data just by saving a file.
- Losing meta-data when doing backup
- When you back up your data, the backup application needs to save and restore meta-data within ADS properly. I did not investigate this issue but my gut feelings are that only a fraction of the tools on the market do consider ADS meta-data and handle them accordingly.
As I've read on reddit, modifying the NTFS tags of a file also modifies its hash sums which renders some workflows that rely on stable content hashes useless.
Summary and Remarks
After being enthusiastic when I found out that Microsoft provides a native file tagging ecosystem with Windows, I had to take a closer look. This enthusiasm was replaced by a disillusion. Everything related to file tagging is hidden from the common user by default. Enabling it results in manual labor not only for the UI but also for each and every file extension separately. Although there are some nice retrieval features for navigation, search does not differ between keywords in tags and keywords anywhere else. It is not entirely clear to me how file-format-specific tags interact with the NTFS tags. Finally, when you did invest some time for tagging files, there is a high chance of losing all this meta-data sometimes without even realizing it.
If Microsoft would act in a way that somebody would be thinking that this tagging feature is ready for production, it would qualify for my bad design decisions series. For me personally, I'd never invest anything in using this feature mainly because of the many ways of losing meta-data without noticing. My current approach for tagging is described on this article. It's an OS-independent and app-independent method with very nice features like TagTrees you can not find elsewhere.
If you would like to get an overview on other non-file-system-based tagging solutions, you can read the bachelor thesis "Marktübersicht von Tagging-Werkzeugen und Vergleich mit tagstore" which can be downloaded at the tagstore page. It's in German language and it reflects the situation of the year 2013.
Before writing this article I needed to implement a necessary feature for my blogging system beforehand. With this, you are now able to click on the screenshot previews to see them with their original size. So this article was in my personal pipe-line for over a year. As a consequence, early findings and screen shots from 2018 are based on Windows 10 Pro version 1803 OS build 17134.165
whereas the most current ones from 2019 are based on Windows 10 Enterprise Evaluation 1809 OS build 17763.805
.
Congratulations for following this very long blog article until its end. I hope I could teach you something on Windows 10 functions and help you decide on its usefulness for your situation. Drop me a line in the comments below when you do have some questions or remarks.