# Using Vim As Your Shell Command-Line Scratch Using (n)vim as your command-line scratch helps you to efficiently fix, create, and run ad-hoc commands. If you are not familiar with vim, I suggest you start playing with the `vimtutor` first. I’ll try my best to explain each keys that I have pressed. In this blog, I’m using [Neovim](https://github.com/neovim/neovim), but you can still follow along even if you’re using [Vim](https://github.com/vim/vim). I have aliased my `nvim` to `vim` with `alias vim=nvim`. Please follow along with the demonstrations, so you can get used to this workflow. ## Fixing Ad-Hoc Commands Have you experienced running a command and realizing that you made a mistake after executing it? You missed some syntax or forgot adding a `sudo`, and much more? ### Commands that requires privilege You are copying a file to another directory, or installing a package, but you forgot that it requires a `sudo` privilege. %[https://youtu.be/q6ZV4yQMye4] ```text I - insert mode before the first character Escape or Ctrl + C - to go to the normal mode ZZ or :wq- to save and exit ``` Note that in vim, keys are case-sensitive, this means that the lowercase `i` is different to the capital `I`. As you have noticed, we were able to edit our previously executed command with fix command (`fc`), it opened the vim editor for us with the command as our content. If we exit from the editor, it’ll run the command that we have edited. For this case, it is better to use sudo-bang-bang (`sudo !!`) if you need to add `sudo` before the previous command. %[https://youtu.be/eq6CV0_d9jg] `!!` - will use your previously executed command If we view the man page of `fc`. It checks for the `FCEDIT` and `EDITOR` environment variables. If both of them are unassigned, it’ll use `vi`. ```text -e ename ... If ename is not given, the value of the FCEDIT variable is used, and the value of EDITOR if FCEDIT is not set. If neither variable is set, vi is used. When editing is complete, the edited commands are echoed and executed. ``` I have exported my `EDITOR` variable in my config file to use neovim. ```bash export EDITOR=nvim ``` You can assign `nano`, or `code` here to use VS Code as your `fc` editor, but I do not recommend it. You’ll see why it’s better to use vim later on. %[https://youtu.be/K7wjiU9dd8o] ### Commands with typo Another use case is if you made a typo in the middle of the command, or missed any quotation marks. **Arrow Keys Spam** %[https://youtu.be/lIAmBhPlZSg] When we try to fix our command, we will spam the thing out of the left arrow key to navigate to the middle of the command to fix it. This workflow is so slow, and even annoying to deal with. **`fc` to the rescue!** %[https://youtu.be/XAw9RP_sQo0] Invoke the `fc` command to fix the previous command that you have executed. This will open a vim editor which we can easily perform search and replace commands. This would be more useful if we have really long commands and arguments. `:s/ooo/o` - perform a substitute `ooo` to `o` `:s/hir/here` - perform a substitute `hir` to `here` For more information about the vim substitute command, please check the help file with `:h :s`. ### Commands with shell expansion Let’s say you want to say hello to the current `USER`. %[https://youtu.be/JHGCmF9nNmo] Hmmmmm 🤔, our `USER` is not printing, but it has value as we have checked it. We have found out that we were using single quotes (`'`), and this means that [shell expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Expansions.html) never happened, what we need is to surround them with double quotes (`"`). We can still use `:s/'/"` here, but we can also leverage the power of vim with plugins! %[https://youtu.be/qk2lxHbRIh0] I have navigated to the text to replace the surrounding quotes, then used [tpope/vim-surround](https://github.com/tpope/vim-surround) plugin. ```text cs'" - it means change surrounding single quote (') to double quote ("). c s ' " ``` If you want to delete a quote, you can use `ds'`, or `ds"` for double quotes. ### Commands that you want to edit without executing We have learned that we can use `fc` to edit or fix our previous command. What if you noticed a typo, and you do not want to execute the command? In that case, what we can do is press `ctrl + a` to move the cursor before the first character in your command line and type pound (`#`), this results into making the command as a comment. %[https://youtu.be/yFt5w1qS-gg] If the command with pound (`#`) in front is executed, it won’t execute the command itself, but instead it’ll result into being a shell comment. The good thing with this is you can use `fc` to open it in your editor and edit it. You can do this because the commented-out command was already added to your command history, and `fc` will edit the previously executed command. Just remove the `#` on your edit. %[https://youtu.be/xGgQLjYbTmc] ### Commands with long parameters If we want to run a postgres container with a command from their documentation, we will run the following command: ```bash docker run -d --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -e PGDATA=/var/lib/postgresql/data/pgdata -v /custom/mount:/var/lib/postgresql/data postgres ``` After we have pasted this in our terminal, it’s hard for us to update the parameter variables. We will use so many arrow key presses just to navigate to the environment variable values and edit it. We can comment this for now and run the `fc` command. By doing this, we can leverage the vim editor to manipulate the text. %[https://youtu.be/XbZ10YsqU4I] `x` - delete the `#` symbol `:s/some-/my-` - substitute the container name ```text /secretciwmyverysecret - '/secret' search and navigate to the first 'secret' word match - 'ciw' delete the inner word, and go into insert mode - 'myverysecret' literal word to replace the password value ``` Add new lines before the `-e` and `-v` ```text :s/ -[ev]/\r\0/g - ':s/' substitute command - ' -[ev]' search for text that has ' -e' or ' -v' - '/\r' add a carriage return - '\0' substitute with the first captured group - '/g' substitute globally, not just the first match ``` With this substitute, we were able to format our command, but there is a problem, in shell with multiple lines, we need to suffix every line with a backslash (`\`). We can achieve that with performing a `normal` command. ```text :%norm A \ - ':%norm' perform a normal command for the whole buffer - 'A' go to insert mode to the end of the line - ' \' add literal space and backslash ``` The `normal` command helps you to perform it on multiple lines as if you’re actually typing it directly on the buffer, to learn more about it, check the help file for `normal` using `:h norm` (`norm` command is just a shorter version of `normal` command). We were now able to update the values of a command with multiple long parameters, and also properly formatted it. If you noticed that my line numbers are not linearly increasing, I’m using a relative numbers, you can learn more about it with `:h relativenumber`. I highly recommend setting `set rnu` config together with `set nu` so you can easily jump to arbitrary lines relative to your current line. ### Commands that you are currently fixing and want to abort it If you have edited your previous command and you do not want to execute it. If you use ZQ or :wq! to exit vim without saving, it’ll still execute your command. %[https://youtu.be/v2XK8w-ke7k] What happened? It turns out that when the editor exits with `0`, it’ll execute your command, and using `:wq!` gracefully exits vim. ```bash echo $? 0 ``` How to solve this? You need to use `:cq` to exit vim with error code. You can add a key bind for this, or you can just delete the content of your vim buffer with `dd` and exit. %[https://youtu.be/gr0FeLG3gTQ] ## Vim Buffers and Shell Fixing your previous command is cool, but what if you want to perform a command operation to the output of your previous command, and update its content? You can use Vim to achieve that. If we perform an operation and pipe it to our Vim buffer, it’ll use the stdin as your buffer content. ```bash echo hello | nvim - # look ma, no dash echo hello | nvim ``` There are times that you need to use dash (`-`) to pipe the standard output to standard input of the next command. In nvim, the dash is not required. ### Sort and remove duplicate names We have a file with a list of names, but there are duplicates. We want to remove the duplicates, and sort them. ```bash cat names.txt John Doe John Doe David William Joseph Thomas David William Joseph Thomas Joseph Thomas ``` We can pipe the output of the `cat names.txt`, or just directly open it with vim. ```bash vim names.txt ``` Then we can now perform a command operation to update the content. ```bash :%!sort -u David William John Doe Joseph Thomas ``` Anything after the `!` character is the shell command. %[https://youtu.be/EMRnCyAWByI] Of course, we do not need to open it in vim to remove the duplicates, it is to simply demonstrate that you can perform commands on the current buffer and replace it with the output of the shell command. ### Sort and remove duplicate names of some part of the buffer We can also specify some area that we want to perform the operation using the visual mode of vim. ```bash cat names2.txt John Doe John Doe David William Joseph Thomas David William Joseph Thomas Joseph Thomas # please do not modify the following lines Christopher Daniel Mark Anthony Christopher Daniel Paul Steven Mark Anthony Paul Steven ``` %[https://youtu.be/9xSjN_y_jVM] First, we need to select the range with the visual line (`V`), and run the `:!sort -u` command. Note that the `'<,'>` characters are automatically added by vim, this represents the selected line range. ### Formatting and filtering JSON content ```bash cat characters.json [ { "id": 1, "name": "Tomoko Kuroki", "age": 15, "height": "147 cm" }, { "id": 2, "name": "Marin Kitagawa", "age": 15, "height": "164 cm" }, { "id": 3, "name": "Kaguya Shinomiya", "age": 17, "height": null } ] ``` This is a JSON file with a content that is not properly formatted. We also want to remove some noise, we only want the `id`, and the `name` properties of the characters. We can do this by using the [jq](https://stedolan.github.io/jq/) command. We can also use any vim plugins like prettier for different file types. %[https://youtu.be/g21jGOxmV10] We have formatted it with `:%!jq .` ```bash :%!jq . [ { "id": 1, "name": "Tomoko Kuroki", "age": 15, "height": "147 cm" }, { "id": 2, "name": "Marin Kitagawa", "age": 15, "height": "164 cm" }, { "id": 3, "name": "Kaguya Shinomiya", "age": 17, "height": null } ] ``` Filtering the properties that we only want with `:%!jq '[.[] | {id, name}]'`. This is really helpful to analyze some JSON data, and want to play with the properties. %[https://youtu.be/mOY9fB_unYY] ```bash :%!jq '[.[] | {id, name}]' [ { "id": 1, "name": "Tomoko Kuroki" }, { "id": 2, "name": "Marin Kitagawa" }, { "id": 3, "name": "Kaguya Shinomiya" } ] ``` Pressing `u` to undo the changes to the buffer. You can learn more about `jq` from their [manual](https://stedolan.github.io/jq/manual/). We were able to easily parse the JSON without leaving our editor! ## Searching file contents You can also pipe the output of your grep to vim, so you can easily navigate to the matched files. If the cursor is under a file path, and pressing `gf` will open another buffer with the file and the cursor is on the line number that was created in our `grep` result. %[https://youtu.be/8LzGbPxIzgU] ```text grep -Hnri 'search keyword here' . | nvim H - print file name n - add line number r - recursive search for each directory i - ignore case ``` Now that we have the matched searches in our buffer with their associated file names and line numbers, we can use `gf` here to navigate to the file under the cursor. This can help you preview and analyze the files, you can get back to the previous buffer with `ctrl + 6`. See `:h buffers` for more information. ## Writing to Shell Can we do it the other way around? Using the content of your buffer to execute them in your shell? One common misconception when you run `vim `, is that your’e editing the actual file, but in fact, you are actually opening a buffer of that file, and not editing the file itself. This is different to the other editors where they will immediately create the file for you. %[https://youtu.be/LNefH_a6JbQ] When you save it with `:w`, you are just overwriting the file with your current buffer, and if the file does not exist, it’ll create it. This is equivalent to `:w %` or `:w yourfile.txt` %[https://youtu.be/3LnOlBfes0Q] With this in mind, what if you write the buffer to your shell? Well...you guessed it right (pun not intended 😉), if you write the buffer to your shell, you can execute commands from your Vim buffer! I am using `zsh` here, but any shell should work. If we write the buffer to a shell, it’ll invoke the command to a sub-shell. ```bash :w !zsh :w !$SHELL ``` ```bash cat file-that-contains-shell-script.txt echo 'i have echo' echo $(pwd) touch newfile.txt ``` %[https://youtu.be/xaPSGL1wrPU] We can also pick which line to execute ```bash :.w !zsh ``` Notice the dot(`.`) before `w`, this means “write the current line of the cursor to the shell”. ```bash cat sample-line.txt echo hello1 echo hello2 echo hello3 echo hello4 echo hello5 ``` %[https://youtu.be/5-04YkJgjB8] ### Shell Level If we print out the `SHLVL` environment variable, we are currently at level `1`. Invoking this command inside vim will result into the level `2`. This means that it is executing it to the sub-shell. %[https://youtu.be/131E9H4shvQ] ### Bindings Configuration Typing `:w !zsh` or `:w !$SHELL` again and again is too much, I recommend adding your own bindings similar to what I have ```bash " write to shell let mapleader=" " nnoremap xs :.w !$SHELL xnoremap xs :.w !$SHELL nnoremap xS :w !$SHELL ``` You can put this into your `.vimrc` file or the `init.vim` for neovim. Then you can just press the leader key which is set to space and `xs`. %[https://youtu.be/XF2hPDsck40] ### File Analysis From our previously found files in the “Searching file contents” section, we decided that we want to delete those files. We need to filter out the file paths first, and there are multiple ways to achieve this. We can use `cut` here, but for now, let’s use `awk` with a colon separator. ```bash :%!awk -F: '{print $1}' ./subfolder0/somefile.txt ./subfolder0/anotherfile.txt ./subfolder1/alpha.txt ./subfolder3/beta.txt ./subfolder4/magic.txt ./sample-line.txt ./sample-line.txt ./sample-line.txt ./sample-line.txt ./sample-line.txt ``` %[https://youtu.be/e59b94-250A] There are duplicate files, let’s perform a unique sort and prepare to delete with `rm`. ```bash :%!sort -u ./sample-line.txt ./subfolder0/anotherfile.txt ./subfolder0/somefile.txt ./subfolder1/alpha.txt ./subfolder3/beta.txt ./subfolder4/magic.txt ``` %[https://youtu.be/olP7tBWj8PY] Now that we have added the rm command on each line, and writing them to your shell will remove the files. ## Shell Intellisense We can improve our workflow by adding intellisense using the [Language Server Protocol (LSP)](https://microsoft.github.io/language-server-protocol/). LSP is a protocol that can help provide some auto completion, quick documentation, go to definition, and many other features. You can use any LSP provider, at the time of this writing, I’m using [coc.nvim](https://github.com/neoclide/coc.nvim) for my LSP. Setting up your own LSP and their configuration is out of scope for this blog, you can check their documentation for it. In order for our LSP to work, there are multiple ways to enable it, but the easiest one I can think of is to set the file type of the buffer that we are currently working on. You can check the current file type with `set filetype?`, and assign a file type using `set filetype=sh`. Now that we have an `sh` filetype, our LSP now knows that we are editing a shell buffer, which will provide us the language features of a shell. ### Command Auto-Completion and Documentation %[https://youtu.be/JPvUm4VYbDk] ### Path Auto Completion %[https://youtu.be/0WVa-ERmRe4] This is just the tip of LSP, and you can check the language server of what features it can offer for you to help in your shell scratch workflow. ## Other Use Cases In this section, I will share some of my workflow that I use day to day. ### Workflows in Git ### Long Branch Names Whenever we create new features, or fixing a bug. It’s better to create new branch from the `main` or any base branch. We usually create branches using the ticket name `feat/ZERO-1234`, but browsing these branches does not provide any context as to what is the purpose of that branch, unless you memorized the ticket number and their description. We can add more details to the branch name, but the problem with this branch names is that the checkouts, rebase, and renames is a bit of pain to type, unless you have auto complete. You can pipe the branch output to vim, and perform the checkouts. ```text feat/ZERO-123-create-some-cool-feature-here-that-no-one-usees fix/ZERO-456-the-bug-that-will-consume-your-days ``` %[https://youtu.be/I9tLevfBEJU] The aliases that you have configured in your Git will work here like `git config --global alias.co checkout`. ### Deleting multiple branches Deleting multiple branches with `grep` is a bit of a hassle, unless you run the `grep` first to see the list of the branches. What we can do is pipe the output to vim, and perform any searches. %[https://youtu.be/-ki6so7KKCY] By doing this, you have a faster feedback of what branches would be deleted, you can even easily remove any branch that was part of the initial `grep`. ### Reading Command Outputs APIs mostly use JSON as their payload. We can easily create them using [jo](https://github.com/jpmens/jo). We can read the command output and put it to your current buffer. For example, we want to create a JSON object with a lower case uuid value for its `id` property, and a simple name. ```bash :read !jo id=$(uuidgen | tr A-Z a-z) name='zaerald' ``` Anything after the `!` will run a shell command. %[https://youtu.be/MQSmpn-hp1M] ## Conclusion You can now efficiently edit previously executed commands using `fc`. Using vim with plugins and LSP can help improve your workflow. Sky is the limit! Using vim as your command-line scratch is much fun to use. Just play with it, and don’t forget to have fun! You can check my dotfiles in [github.com/zaerald/dotfiles](https://github.com/zaerald/dotfiles). Please note that my dotfiles continues to evolve, and there’s a higher chance that it is different now compared at the time of this writing.