CLOSED: [2017-01-01 Sun 21:31] SCHEDULED: <2017-01-01 Sun>
:PROPERTIES:
:CREATED: [2017-01-01 Sun 20:46]
:ID: 2017-01-01-memacs-grep
:END:
:LOGBOOK:
- State "DONE"       from "NEXT"       [2017-01-01 Sun 21:31]
:END:

[[https://github.com/novoid/Memacs][Memacs]] is a nice Python software that collects misc data and generates
Org-mode files for me. This way, my agenda contains all of my emails,
text messages, Git commits, photographs I took, and so forth.

With the Memacs-module [[https://github.com/novoid/Memacs/blob/master/docs/memacs_filenametimestamps.org][filenametimestamps]], all of my files whose
file name begins with an [[http://www.cl.cam.ac.uk/~mgk25/iso-time.html][ISO time-stamp]] are indexed and stored in an
Org-mode file.

Using a custom link, I am able to define and access links to those
files without the need of knowing their location in my file system
hierarchy:

#+BEGIN_SRC elisp
(setq org-link-abbrev-alist
      '(
        ;; tsfile = time-stamp file
	("tsfile" . "~/path/to/memacs/files.org_archive::/\*.*%s/")
	))
#+END_SRC

Therefore, this works with the standard Org-mode featureset: you
create a link which looks like this:

: [[tsfile:2017-01-01_Manual_for_my_amplifier.pdf]]

or

: [[tsfile:2017-01-01_Manual_for_my_amplifier.pdf][My amp manual]

When you want to open the PDF file, you put the cursor on the link and
press =C-c C-o= (for =org-open-at-point()=). Emacs opens the
=files.org_archive= in a buffer and jumps right to the heading of this
file. The heading consists of an absolute link to the file in your
file system. Therefore, when you press =C-c C-o= once again, your PDF
file opens before your eyes.

This method is totally independent of the folder that holds the PDF
file. I can move around those files in my file system hierarchy
without getting broken links.

Neat, isn't it?

In my case, I got so many files containing ISO datestamps, that my
=files.org_archive= holds almost half a million entries. Opening this
Org-mode file with my Emacs settings and jumping to the corresponding
heading takes very long. So long that it is not feasible to be used
any more.

Behold, [[https://en.wikipedia.org/wiki/Grep][grep]] to the rescue!

-----------------------

An [[http://kitchingroup.cheme.cmu.edu/blog/2016/11/04/New-link-features-in-org-9/][awesome blog article by John Kitchin about the new link features of
Org-mode version 9]] let me think of an alternative approach: Why not
use the powerful (and fast) =grep= tool to query for the matching
headings, extract the absolute path from it, and open the linked file?

Well, I tried to code this Elisp snippet myself and failed - as usual.
Simple things that I'd code in Python within minutes are tedious to
impossible for me in Elisp. Oh, how I pity myself for not being able
to do those things in Elisp myself.

To my great relief, John Kitchin coded the lines necessary so that I
am able to present you my currently working solution for the issue:

#+BEGIN_SRC elisp
(defvar memacs-root "~/org/memacs/")
(defvar memacs-file-pattern "files.org_archive") ;; also possible: "*.org"

;; by John Kitchin
(defun my-handle-tsfile-link (querystring)
  ;; get a list of hits
  (let ((queryresults (split-string
                       (s-trim
                        (shell-command-to-string
                         (concat
                          "grep \""
                          querystring
                          "\" "
                          (concat memacs-root memacs-file-pattern))))
                       "\n" t)))
    ;; check length of list (number of lines)
    (cond
     ((= 0 (length queryresults))
      ;; edge case: empty query result
      (message "Sorry, no results found for query: %s" querystring))
     (t
      (with-temp-buffer
        (insert (if (= 1 (length queryresults))
                    (car queryresults)
                  (completing-read "Choose: " queryresults)))
        (org-mode)
        (goto-char (point-min))
        (org-next-link)
        (org-open-at-point))))))

(org-link-set-parameters
 "tsfile"
 :follow (lambda (path) (my-handle-tsfile-link path))
 :help-echo "Opens the linked file with your default application")
#+END_SRC

Thanks John! This brings me back a missed feature to my Org-mode life.