Document is not exported when using Emacs in batch-mode via --batch

[jgru]

Dear @kawabata,

first of all, thank you for providing ox-pandoc.
I discovered that exporting with ox-pandoc (tested with org-pandoc-export-to-docx) is not working when running Emacs in batch-mode via --batch.

To reproduce this you could run the following snippet from a directory containing at least one org-file:

emacs --batch --load custom-export.el --funcall batch-export

whereas custom-export.el contains the following function-definition:

(defun batch-export ()
  "docstring"
  (mapc
    (lambda (f) (interactive "")
      (with-current-buffer
        (switch-to-buffer (find-file-noselect f))
        (org-pandoc-export-to-docx)
        (kill-buffer)))
    (directory-files "./" t ".org$")))

This produces the following output:

Running pandoc with args: (-f org -t docx -o /abs/path/to/file.docx /abs/path/to/file.tmpK2cf53.org)

However, the log message Exported to file.docx is missing, and pandoc seems to be never actually invoked.
Do you have any idea what the issue might be here?

It would be great if ox-pandoc could be used in batch-mode as well.

Best regards,
jgru

[antler5]

Hmm... I got a little further by opening stdin (ie. echo y | emacs [...]), the final file was created (not just the tempfile), but the content was still incomplete (no babel blocks) and the tempfile was never removed. I added a loop that waits for pandoc to exit[1], and now the tempfile gets deleted and I get the "Exported to [...]." feedback, but the content is still incomplete.

(progn
  (require 'ox-pandoc)
  (org-mode)
  (setq org-confirm-babel-evaluate nil) ; doesn't help
  (org-pandoc-export-to-commonmark nil nil t)
  (while (process-list)
    (with-temp-file "/dev/stderr"
      (insert (format "%s" (process-list))))
    (sit-for 1)))

May or may not have time for a deeper look tomorrow.

---

1: Piping to stdin no longer makes a difference

[antler5]

Got it! I'm not super familiar with the org-mode / org-babel codebases, and in hindsight there were probably better ways of doing this, but every once in a while you just get stuck on a lil' thing.

What I did was insert print-debug statements on each end of the chain-of-execution-- I knew that each started in roughly the same environment, and that at some point something diverged. So I started sticking print statements in the middle, watching argument values and checking the buffer state to bisect my way towards the point of divergence, descending into deeper function definitions as I went:

org-pandoc-export
-> org-export-to-file (ox-pandoc.el)
-> org-export-as (ox.el)
-> org-babel-exp-process-buffer (ob-exp.el)
-> org-babel-exp-src-block (ob-exp.el)
-> org-babel-exp-do-export (ob-exp.el)
-> org-babel-exp-results (ob-exp.el)
-> ;; Skip code blocks which we can't evaluate.

All told I followed the path of execution down through about 750 LOC, never once stopping to adjust my tabwidth to resolve the inconsistent indentation: I didn't want to put in the time to understand everything I was seeing anyways. It took maybe a bit over 2 two hours? Idk, I stopped to shower, etc. I knew I'd found what I was looking for with that comment: org-babel silently skips code blocks when org-babel-execute:<lang> is un-bound, and (as documented) not even org-babel-execute:sh is bound in batch-mode unless you eg. (require 'ob-shell).

What I'm asking myself is: was there an easier way? Like, maybe less bisection would have necessary if I'd guessed what functions to read further into, or checked the end of each function to see if the path plausibly continued down into a function call from there. But I believed I had to be ready to find the argument, variable, or branch responsible at any position and on any level, least I overlooked it and lost even more time. Guessing is a risk, would have strained me to keep track of more information in an unfamiliar environment, and I know well that the most effective way to solve a bisect-able issue is to just commit to resolving the bisection with steady, algorithmic confidence (automatically, if at all possible). I probably should have checked the *Warnings* buffer, but it wouldn't have helped; babel skips these blocks without printing any warnings. Maybe it should? But AFAIK the behavior is documented.

So I don't think I could have reliably done this any faster, and that disappoints me because I won't always be able to spare so much time (or more mental energy) for a "small" issue (whatever that means). I think the only way to develop a stronger approach for this sort of obscure bug-dive is to become more comfortable making guesses, to spare more mental energy (perhaps time!?) becoming familiar with and keeping track of the code-base (ie. adjusting indentation settings, maybe even some light refactoring to help grok the structure and control-flow), and building a more robust "mental VM" to be able to time into more definitions and spend more time reading ahead without executing to verify-- but that situation of reading and guessing without as regularly verifying feels to me like fertile ground for misunderstanding, overlooking, and getting lost, when stacked against an equally time-consuming but inevitably robust bisection resolution. Idk, I just don't wanna do that again. Hope future me cracks this sort of shit.

[antler5]

Oh, and you gotta disable the confirmation prompt.
So to give an example for your code as-asked:

(defun batch-export ()
  "docstring"
  (setq org-confirm-babel-evaluate nil)
  (require 'ob-shell)
  ;; ^ or any other langs you might need.
  ;; Calling org-babel-do-load-languages should work as well:
  ;; (org-babel-do-load-languages
  ;;   'org-babel-load-languages
  ;;   '((shell . t)))
  (mapc
    (lambda (f)
      (interactive "")
      (with-current-buffer
          (switch-to-buffer (find-file-noselect f))
        (org-pandoc-export-to-docx)
        (while (seq-filter (lambda (p)
                             (equal (process-name p) "pandoc"))
                           (process-list))
          (sleep-for 0 15))
        (kill-buffer)))
    (directory-files "./" t ".org$")))

and I think we can say issue closed