UOMF: Path-Independent Links to Local Files Via Locate

This is an article from a series of blog postings. Please do read my "Using Org Mode Features" (UOMF) series page for explanations on articles of this series.

• Update 2022-02-11:
  • Why Not Use Attachments?
  • Why Using Local Files?
  • How I Generate Static File Links

For linking to local files, I have already written about my tsfile: custom link method using Memacs. I use this for managing local files such as images and it is also an important help for my blogging system.

Here, I want to explain a simpler version for people who don't use Memacs for indexing files (yet).

The biggest advantage of using Org-mode links based on that concept is that you only have to provide the basename of a file without its storage path and still get a successful retrieval process as long as you use unique file names. This way, you can move around files in your file system and rename directories without breaking any links in your Org-mode. How cool is that?

Org-mode provides a method to associate reference material with an outline node via attachments.

I don't use this handy Org-mode feature except for some rare cases where my Org heading structure corresponds with my file system structure. While the fact that my Org hierarchy differs from my file hierarchy could be discussed separately (there are good arguments for both sides), this is my situation for now. This way, org attachments do not provide me much advantage to this method of linking local files while it does give me freedom to move around things without breaking stuff.

I do care a lot about local files. I will continue to curate a local file collection that contains my digital photographs, my scanned paper documents, the files I generate for various purposes, and so forth.

You can read about my opinion on using the public cloud and what you need to consider before starting to give away your data to those services such as Apple, Google, Microsoft, Dropbox and you name it.

Before I explain how those robust file links are done, I would like to mention what my alternative looks like when I create classic static file links.

1. I usually invoke C-z which is bound to my-dired-recent-dirs() from my Emacs configuration.
   • It was briefly mentioned in my complex folder article and deserves more attention. It allows me to jump to recently and frequently used directories very efficiently.
2. I chose the file to link, usually by invoking the file filter via /
3. Invoke one of my two hydra-provided functions to generate a file link.
4. Switch back to my Org-mode buffer and yank the newly generated link.

To actually generate a file link, I do have two different functions at hand: my-dired-copy-filename-as-absolute-link() which is called "absolute link" in my hydra-dired() returns something like that:

 [[file:/home/user/dir1/filename.pdf]]	  

Furthermore, "Absolute basename" from hydra-dired() returns something like that:

 [[file:/home/user/dir1/filename.pdf][filename.pdf]]	  

As you can see, those links do break when the file is moved to a different directory or any directory within its path is renamed.

Therefore, I usually do link files that have unique file names with a different method I want to explain in the following sections.

Now let's see how this is implemented.

The method to retrieve local files here is using the locate command, UNIX-like systems do provide out of the box. In simple words, the updatedb command indexes filenames of all local files once a day. This simple index can be queried using the locate command. A command line query looks like that:

 locate "2022-02-10 report"	  

... or sometimes:

 locate 2022-02-10 | grep -i report	  

This way, I do retrieve most files when I can not remember their storage path. It's my poor man's desktop search if you will. It is noteworthy that it doesn't index or query file content, just the file name.

If you're running GNU/Linux or macOS, this index comes "for free". If you want to use that index to link and retrieve local files, you might want to implement the method explained here.

The method described here is using the custom link definition method introduced with Org 9.0 as explained by John Kitchin here.

For the link name, I chose lfile: which stands for "local file" or "locate file" - whatever you prefer.

An example link looks like that:

 [[lfile:2022-02-10 business report -- final.pdf]]	  

With a description, it looks like that:

 [[lfile:2022-02-10 business report -- final.pdf][Our business report from Q1 2022]]	  

If I do invoke org-open-at-point (usually via C-c C-o) when my point is above such a link, my system-defined application that handles the file extension is opened with that file. In this case, my default PDF viewer shows me the report.

First, we do need a method to handle the link:

(defun my-handle-lfile-link (querystring)
  ;; get a list of hits
  (let ((queryresults (split-string
                       (s-trim
                        (shell-command-to-string
                         (concat
                          "locate \""
                          querystring
                          "\" "
                          )))
                       "\n" t)))
    ;; check length of list (number of lines)
    (cond
     ((= 0 (length queryresults))
      ;; edge case: empty query result
      (message "Sorry, no results found for query: %s" querystring))
     ((= 1 (length queryresults))
      ;; exactly one hit:
      (my-open-in-external-app (car queryresults))
      )
     (t
      ;; in any other case:
      (message "Sorry, multiple results found for query: %s" querystring)
      ;; FIXXME: ask user to select among multiple hits.
      )
      )))	  

If you do take a look at my Emacs configuration, you can also find my-handle-tsfile-link() which asks you to choose among multiple entries when the search result is not unique. I omitted this here because I link only unique file names myself which is mostly ensured by applying date2name on file names.

You can't use the method above directly without either using my-open-in-external-app() which you can also find in my configuration or using a method of choice to open arbitrary files.

For that purpose, you can find ideas here:

• find-file is the commend to open it in GNU Emacs directly. So you can just replace "my-open-in-external-app" with "find-file" if you prefer an Emacs-only experience.
  • Visiting-functions explain you more about that set of functions.
• Xah Lee has an article on opening files in external apps if you prefer the system applications approach without re-using my method.

Now that we do have a function that is able to handle the file links, we do have to tell Org-mode to use that method for links with the abbreviation lfile:.

This is done via the following snippet:

(org-link-set-parameters
 "lfile"
 :follow (lambda (filename) (my-handle-lfile-link filename))
 :help-echo "Opens the file located via \"locate\" with your default application"
 :face '(:foreground "DarkSeaGreen" :underline t)
)	  

You can adjust the face for the link to your liking. If you take a look at my configuration, I do maintain different classes of link colors for local and remote links (search for "link colors") to keep the color scheme small and recognizable.

In my case, I had to restart GNU Emacs in order to make sure that everything is loaded properly and I could start using my new custom links.

The most commonly used snippet system might be YASnippet. I prefer an extension for it named yankpad which simplifies the management of the snippets by using Org-mode files for it.

With yankpad, I defined the following heading to add a new link which I usually do have in my clipboard:

 ** lfile: lfile
 [[lfile:$1][${2:$$(unless yas-modified-p
  (let ((field (nth 0 (yas--snippet-fields (first (yas--snippets-at-point))))))
    (concat (buffer-substring (yas--field-start field) (yas--field-end field)))))}]] $0	  

You can use a simpler definition. Mine does generate something like ...

 [[lfile:foo][foo]]	  

... by only providing "foo" once.

Therefore, my process for adding a link looks like that:

1. Get the basename of the file in my (system) clipboard.
2. In Org-mode, type lfile and press the binding for yankpad-expand to invoke yankpad.
3. Yank the basename and proceed with the tabulator key.

The new link is inserted at the current point. Opening the linked file is as easy as invoking C-c C-o onto the link or using a mouse click.

My main use of lfile: links is on my business machine where I don't use or need Memacs at the moment. Since the whole storage is on a fast SSD, updatedb only runs for about ten seconds. You can check this on your machine if you run sudo time updatedb yourself.

Having such a fast update process at hand, you can decide to go from a daily update - which is the default - to an hourly update.

For that, I just had to move the execution script on my Ubuntu machine from one directory to another:

 sudo mv /etc/cron.daily/locate /etc/cron.hourly/	  

If your system has a different method to invoke the database update, please refer to your system documentation.

That's it.

I hope you could follow my idea and the instructions so that you're able to profit from this neat method yourself. I can not think of not using this method any more. It offers so much advantages to me on an daily basis.

Please add comments below if you want to ask me something or if you do have an add-on idea.