Emacs: Use Different Search Methods Depending on Number of Lines

The issue: Swiper is a great search method. However, for large files, it takes very long from search process invocation to be actually able to enter the search query. I added to a bug report on Github for this issue.

Until this issue is fixed, I have to think about a workaround.

What are the alternative search methods? Before Swiper, I was using incremental search (isearch-forward). It might be the case that search-forward is even faster. But I won't settle for less than isearch. A different alternative would be occur. For now, standard incremental search is sufficient for large files.

So: how about switching search method of C-s depending on the number of lines of the current buffer? Small files gets the advanced Swiper search, large files gets the basic incremental search.

Let's do it.

To prevent me from chasing the wrong things, I did a profiler-run:

Function                                                  CPU Samples    %
- command-execute                                               34706  98%
 - call-interactively                                           34706  98%
  - swiper                                                      34618  98%
   - apply                                                      34618  98%
    - compiled 0x156eb5d                                        34618  98%
     - swiper--ivy                                              34618  98%
      - swiper--candidates                                      32825  93%
       - replace-regexp-in-string                                  88   0%
          apply                                                    24   0%
        + funcall                                                   8   0%
      + ivy-read                                                  196   0%
  + byte-code                                                      49   0%
  + minibuffer-complete                                            31   0%
  + execute-extended-command                                        8   0%
+ ...                                                             452   1%
+ yas--post-command-handler                                        16   0%
+ timer-event-handler                                               6   0%
+ redisplay_internal (C function)                                   4   0%	  

OK, the culprit seems to be swiper--candidates.

Then, I had to find the sweetspot. How many lines does a buffer have until Swiper gets slow? For this purpose, I did some very basic performance measurements from the invocation of the search command until I am really able to enter my search query:

It seems to be the case at approximately 20,000 lines. Here, swiper-20160124.429 from elpa gets slow on my intel i5 with GNU Emacs 24.4.1 on Debian GNU/Linux.

Now we have found out more about the background, let's continue with coding our workaround.

As an Elisp amateur, I always get a bit nervous when I have to actually generate Elisp code. But this should be an easy one even for me.

Counting lines of a buffer works quite fast in large files:

(count-lines (point-min) (point-max)))	  

This is how the definition of two different search methods looks like:

(global-set-key "\C-s" 'swiper)          ;;fancy search
;; ... OR ...
(global-set-key "\C-s" 'isearch-forward) ;;normal search	  

This is my workaround that switches the search method according to the number of lines in the current buffer:

  (defun my-search-method-according-to-numlines ()
    "Determines the number of lines of current buffer and chooses a search method accordingly"
    (interactive)
    (if (< (count-lines (point-min) (point-max)) 20000)
        (swiper)
      (isearch-forward)
      )
    )
  (global-set-key "\C-s" 'my-search-method-according-to-numlines)	  

While I was working on my workaround above, Oleh Krehel answered to my GitHub issue with his solution to the same issue:

(defun ora-swiper ()
  (interactive)
  (if (and (buffer-file-name)
           (not (ignore-errors
                  (file-remote-p (buffer-file-name))))
           (if (eq major-mode 'org-mode)
               (> (buffer-size) 60000)
             (> (buffer-size) 300000)))
      (progn
        (save-buffer)
        (counsel-grep))
    (swiper--ivy (swiper--candidates))))

(global-set-key "\C-s" 'ora-swiper)	  

This is clearly more elegant than mine. He checks on buffer size, distinguishes Org-mode buffers from the rest, and uses other (advanced) search methods such as counsel.

Unfortunately, I got issues with counsel-grep and his swiper command:

(swiper--ivy (swiper--candidates)) results in (wrong-type-argument stringp (#( followed by a string containing the whole buffer.

counsel-grep results in Symbol's function definition is void: ivy-set-display-transformer. I found #404 but could not resolve the issue with it.

Another helpful comment by Oleh and I found out that most of my Emacs packages were /very/ outdated due to a misconfigured package-archives setting. After a huge update session, counsel as well as swiper are doing great!

The performance boost is astonishing!

There is a reddit-thread on this topic.