> For the complete documentation index, see [llms.txt](https://docs.glesys.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.glesys.com/products/storage/object-storage/how-tos/clients-and-tooling/backups-using-restic.md). # Backups using Restic *** We all know how important backup is, yet a secure and reliable backup solution is often missing. Some may have external hard drives or a small home NAS, but what good does that do in the event of theft or fire? Enter Restic, a modern backup tool that lets us create encrypted backups. The program is entirely free and open‑source, and it also uses a technique called *deduplication*, which means that only unique data is stored in the backup. This helps keep backup sizes low. Since version 0.14.0, Restic also supports compression. Additionally, you get a complete history of the repository. Each new backup becomes a new *snapshot*. That means you can restore your data as it looked three months ago—or even as far back as two years ago. Here, we'll use Glesys Object Storage to store the Restic backup data. This gives you a backup that is stored off‑site. ## Install Restic in Linux and macOS The installation procedure is the same for Linux and macOS. Start by downloading the correct release of Restic from [GitHub](mailto:undefined). The latest version is listed at the top. Under the Assets heading, choose the release that matches your computer and operating system. For example, if you have an Intel‑based Mac, select the file whose name ends with `darwin_amd64.bz2`. If you have an M1‑ or M2‑based Mac, select `darwin_arm64.bz2`. For most Linux distributions running on a typical PC, choose the file whose name ends with `linux_amd64.bz2`. Download the file to the *Downloads* directory on the computer via the browser. Next, you need to unpack the file and place it in the proper directory on the computer. You do this with the following commands. In the example below, we have downloaded version `0.15.2` of Restic for Linux for standard PCs. {% code title="Multiple commands" %} ``` cd ~/Downloads bunzip2 restic_0.15.2_linux_amd64.bz2 sudo mkdir -p /usr/local/bin sudo mv restic_0.15.2_linux_amd64 /usr/local/bin/restic sudo chmod +x /usr/local/bin/restic ``` {% endcode %} Verify that the `restic` command is working. {% code title="Command" %} ``` restic version ``` {% endcode %} This should output something like the text below. {% code title="Output" %} ``` restic 0.15.2 compiled with go1.20.3 on linux/amd64 ``` {% endcode %} ## Create an object storage instance See the section on [Create an instance](/products/storage/object-storage/how-tos/create-an-instance.md) for instructions on how to create an object storage instance. Make a note of the access key, the secret key, the endpoint address, and the name of the initial bucket. ## Initialize a Restic repository Now it’s time to create and initialize a Restic repository. This step creates a directory structure, along with encryption keys and other necessary files in the bucket. When working with Restic repositories, it’s easiest to set all settings as environment variables. That way, you don’t have to type them repeatedly. Environment variables also make it simpler later on when you automate the backups. Here, start by disabling shell history so that your key doesn’t appear in the `history` file of your shell. {% code title="Multiple commands" %} ``` unset HISTFILE export RESTIC_REPOSITORY="s3:https://objects.dc-fbg1.glesys.net/cool-recipe" export AWS_ACCESS_KEY_ID="xxxxxx" export AWS_SECRET_ACCESS_KEY="zzzzzz" export RESTIC_PASSWORD="debit4scotland3hillock3tillage" ``` {% endcode %} `RESTIC_REPOSITORY` is where the backup will be stored. It starts with *s3* because Glesys object storage is S3‑compatible. Then comes the address `https://objects.dc-fbg1.glesys.net/` (*dc-fbg1* is the Falkenberg data centre). This is the address that was shown when you created your object storage instance. After the final slash, the name of your initial bucket is appended, which in this case is *cool-recipe.* `AWS_ACCESS_KEY_ID` is the key that was displayed under *Access Key* when you created the storage instance. `AWS_SECRET_ACCESS_KEY` is the *secret key* that was shown when you created the storage instance. `RESTIC_PASSWORD` is the encryption password for the Restic repository. This is a password you choose yourself. It’s crucial to remember this password; without it, you will never be able to access the backup again. It’s also important to choose a long password with many different characters. Now that all the values are ready, you can initialise the repository with the command `restic init`. {% code title="Command" %} ``` restic init ``` {% endcode %} The command will output something similar to the example below. {% code title="Output" %} ``` created restic repository 930a936a8d at s3:https://objects.dc-fbg1.glesys.net/cool-recipe Please note that knowledge of your password is required to access the repository. Losing your password means that your data is irrecoverably lost. ``` {% endcode %} ## Making backups and working with repositories You have now prepared everything needed to start backing up your files. In this example, we will back up the `Documents` directory in the user’s home directory. {% code title="Command" %} ``` restic backup ~/Documents ``` {% endcode %} This will output something like below: {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) created new cache in /home/jake/.cache/restic no parent snapshot found, will read all files Files: 11 new, 0 changed, 0 unmodified Dirs: 3 new, 0 changed, 0 unmodified Added to the repository: 12.834 MiB (12.298 MiB stored) processed 11 files, 37.326 MiB in 0:03 snapshot bbad22da saved ``` {% endcode %} The backup is now complete. You can also see that compression and deduplication worked: the files in the backup occupy 12.8 MiB, even though the files on disk total 37.3 MiB. You can verify that everything looks correct before proceeding. First, list all *snapshots* contained in your Restic repository: {% code title="Command" %} ``` restic snapshots ``` {% endcode %} This will output a list of the current snapshots in the repository: {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) ID Time Host Tags Paths ------------------------------------------------------------------------------------- bbad22da 2023-05-30 10:53:51 red-dwarf.nixnet.test /home/jake/Documents ------------------------------------------------------------------------------------- 1 snapshots ``` {% endcode %} Each snapshot is labeled with the computer the backup originates from (here *red‑dwarf.nixnet.test*) and the directory that was backed up. You can also list the files that a particular snapshot contains. To do this, use the snapshot’s ID number, which in this case is *bbad22da* (you can see it under the *ID* column in the output from the command above). {% code title="Command" %} ``` restic ls bbad22da ``` {% endcode %} {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) snapshot bbad22da of [/home/jake/Documents] filtered by [] at 2023-05-30 10:53:51.670098007 +0200 CEST): /home /home/jake /home/jake/Documents /home/jake/Documents/cat1.jpg /home/jake/Documents/cat2.jpg /home/jake/Documents/cat3.jpg /home/jake/Documents/coding.mp4 /home/jake/Documents/glitch.mp4 /home/jake/Documents/lorem-ipsum1.txt /home/jake/Documents/lorem-ipsum2.txt /home/jake/Documents/pet-rat.jpg /home/jake/Documents/pet-rat2.jpg /home/jake/Documents/pet-rat3.jpg /home/jake/Documents/test.data ``` {% endcode %} If you modify a file, for example `lorem‑ipsum1.txt`, and then create a new backup, only that file—and only the changes—will be transferred to save storage space. The previous snapshot will be used for the files that haven’t changed. {% code title="Multiple commands" %} ``` echo "That is the end of the story" >> ~/Documents/lorem-ipsum1.txt restic backup ~/Documents ``` {% endcode %} {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) using parent snapshot bbad22da Files: 0 new, 1 changed, 10 unmodified Dirs: 0 new, 3 changed, 0 unmodified Added to the repository: 10.239 KiB (2.913 KiB stored) processed 11 files, 37.326 MiB in 0:01 snapshot 5f80356f saved ``` {% endcode %} Here you see that one file has been changed and ten remain unchanged. This snapshot occupies only three KiB. You now have two snapshots in the repository: {% code title="Command" %} ``` restic snapshots ``` {% endcode %} {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) ID Time Host Tags Paths ------------------------------------------------------------------------------------- bbad22da 2023-05-30 10:53:51 red-dwarf.nixnet.test /home/jake/Documents 5f80356f 2023-05-30 11:36:00 red-dwarf.nixnet.test /home/jake/Documents ------------------------------------------------------------------------------------- 2 snapshots ``` {% endcode %} If you list the files for the new snapshot, you will see all the files, including those that haven’t changed. {% code title="Command" %} ``` restic ls 5f80356f ``` {% endcode %} {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) snapshot 5f80356f of [/home/jake/Documents] filtered by [] at 2023-05-30 11:36:00.525541336 +0200 CEST): /home /home/jake /home/jake/Documents /home/jake/Documents/cat1.jpg /home/jake/Documents/cat2.jpg /home/jake/Documents/cat3.jpg /home/jake/Documents/coding.mp4 /home/jake/Documents/glitch.mp4 /home/jake/Documents/lorem-ipsum1.txt /home/jake/Documents/lorem-ipsum2.txt /home/jake/Documents/pet-rat.jpg /home/jake/Documents/pet-rat2.jpg /home/jake/Documents/pet-rat3.jpg /home/jake/Documents/test.data ``` {% endcode %} ## Restore files from a backup It's time to test restoring the `Documents` directory to a temporary location. Restore the files from the latest snapshot, which in this example is *5f80356f* as seen in the previous section. The backup will be restored to the directory `/tmp/restic‑lab`. {% code title="Command" %} ``` restic restore --target /tmp/restic-labb 5f80356f ``` {% endcode %} {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) restoring to /tmp/restic-labb ``` {% endcode %} The `home/jake/Documents` directory has now been restored under `/tmp/restic‑lab/`. List the files to make sure everything looks correct: {% code title="Command" %} ``` ls -l /tmp/restic-labb/home/jake/Documents/ ``` {% endcode %} {% code title="Output" %} ``` total 38240 -rw-r--r-- 1 jake jake 320729 may 28 18:47 cat1.jpg -rw-r--r-- 1 jake jake 291208 may 28 18:47 cat2.jpg -rw-r--r-- 1 jake jake 150735 may 28 18:47 cat3.jpg -rw-r--r-- 1 jake jake 6467086 may 30 10:48 coding.mp4 -rw-r--r-- 1 jake jake 5073064 may 30 10:48 glitch.mp4 -rw-r--r-- 1 jake jake 1781 may 30 11:35 lorem-ipsum1.txt -rw-r--r-- 1 jake jake 1822 may 28 18:44 lorem-ipsum2.txt -rw-r--r-- 1 jake jake 331545 may 28 18:46 pet-rat2.jpg -rw-r--r-- 1 jake jake 123126 may 28 18:46 pet-rat3.jpg -rw-r--r-- 1 jake jake 163475 may 28 18:46 pet-rat.jpg -rw-r--r-- 1 jake jake 26214400 may 30 10:50 test.data ``` {% endcode %} The new line we added to `lorem‑ipsum1.txt` is also present. {% code title="Output" %} ``` atque ex accusantium dolore ut quis minus sit aliquam fugit est mollitia autem! That is the end of the story ``` {% endcode %} ### Restore individual files It is also possible to restore individual files from a snapshot. This is done using the `--include` flag. Here, let's restore the file `lorem‑ipsum1.txt` from the first snapshot—before we added the new line—to the directory `/tmp/new‑restic‑lab`, which results in the file being placed at `/tmp/new‑restic‑lab/home/jake/Documents/lorem‑ipsum1.txt`. 
restic restore --target /tmp/new-restic-lab --include /home/jake/Documents/lorem-ipsum1.txt bbad22da
ls -l /tmp/new-restic-lab/home/jake/Documents/
total 4
-rw-r--r-- 1 jake jake 1759 may 28 18:44 lorem-ipsum1.txt
tail -n 2 /tmp/new-restic-lab/home/jake/Documents/lorem-ipsum1.txt
atque ex accusantium dolore ut quis minus sit aliquam fugit est mollitia autem!
## Automate the backup Restic is designed to be easy to script and automate, which is evident. Here, we’ll create a simple Bash script to back up the documents directory we’ve been working with earlier. Then we’ll add the script to the user’s *crontab* so the directory is backed up automatically every day. Create a file named `document-backup.sh` in your home directory with the following contents. Adjust the variables and paths to match your own setup. The variables are the same ones we used previously. {% code title="document-backup.sh" %} ```bash #!/bin/bash export RESTIC_REPOSITORY="s3:https://objects.dc-fbg1.glesys.net/cool-recipe" export AWS_ACCESS_KEY_ID="xxxxxx" export AWS_SECRET_ACCESS_KEY="zzzzzz" export RESTIC_PASSWORD="debit4scotland3hillock3tillage" /usr/local/bin/restic backup /home/jake/Documents ``` {% endcode %} Then, change the file’s permissions so that it becomes executable. Also, adjust the permissions so that only your own user can read and write the file, protecting the keys and password. {% code title="Command" %} ``` chmod 700 document-backup.sh ``` {% endcode %} Let's run the script to verify it's working. {% code title="Command" %} ``` ./document-backup.sh ``` {% endcode %} It should output something like this if it's working. {% code title="Output" %} ``` repository 930a936a opened (version 2, compression level auto) using parent snapshot 5f80356f Files: 0 new, 0 changed, 11 unmodified Dirs: 0 new, 2 changed, 1 unmodified Added to the repository: 734 B (583 B stored) processed 11 files, 37.326 MiB in 0:02 snapshot e640c72f saved ``` {% endcode %} Everything worked, and no files have changed since the previous snapshot. Now, add the script to the user’s crontab: {% code title="Command" %} ``` crontab -e ``` {% endcode %} Now a text editor opens, most likely `nano` or `vi`. Add the following line, adjusting the path to match your user’s home directory. Then exit the editor and save the file. {% code title="crontab" %} ``` 5 21 * * * /home/jake/document-backup.sh ``` {% endcode %} Now the `Documents` directory will be backed up automatically every evening at 21:05. ### Add a log file If your computer or server has email enabled, you will also receive an email containing all the output from the cron job execution. This way, you can verify that everything worked as intended. If you instead want to log the latest run to a file, you can modify the crontab line so that it looks like this: {% code title="crontab" %} ``` 5 21 * * * /home/jake/document-backup.sh > /home/jake/restic-run.log 2>&1 ``` {% endcode %} After 21:05, you can view the output from the most recent run in the file `restic-run.log`. {% code title="Command" %} ``` cat restic-run.log ``` {% endcode %} {% code title="Output" %} ``` using parent snapshot e197a21f Files: 0 new, 0 changed, 11 unmodified Dirs: 0 new, 2 changed, 1 unmodified Added to the repository: 734 B (583 B stored) processed 11 files, 37.326 MiB in 0:01 snapshot ba27b049 saved ``` {% endcode %} ## Real-world deduplication Depending on the type of data you back up, Restic’s deduplication can be highly effective. I recently backed up the disks of four virtual machines. All of the VMs were running Debian 11. Together, those four disks occupied 122 GiB on the local drive. The backup in Restic, however, took only 17.7 GiB. Because all the VMs were so similar, there was a lot of redundant data that deduplication could ignore. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.glesys.com/products/storage/object-storage/how-tos/clients-and-tooling/backups-using-restic.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.