Skip to content

Commit 7087e41

Browse files
author
Jean Cochrane (Lead developer, DataMade)
committed
Migrate docs on GPG and Blackbox
1 parent b61f99e commit 7087e41

File tree

3 files changed

+302
-1
lines changed

3 files changed

+302
-1
lines changed

‎README.md

+3-1
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,9 @@ _In alphabetical order and including links to external repository-based document
3636
- [Scraping](scraping/)
3737
- [`lxml` for web scraping](scraping/lxml-for-web-scraping.md)
3838
- [Searching data](search/)
39-
- [Security](https://bit.ly/cryptochecklist)
39+
- [Security](security/)
40+
- [GPG and Blackbox](security/gpg/blackbox.md)
41+
- [Crypto checklist](https://bit.ly/cryptochecklist)
4042
- [Software testing](https://github.com/datamade/testing-guidelines)
4143
- [The shell and Ubuntu](shell/)
4244
- [tmux, best practices](shell/tmux-best-practices.md)

‎security/README.md

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Security
2+
3+
This directory records best practices for keeping your web apps and your work secure.
4+
5+
## Guides
6+
7+
- GPG and Blackbox
8+
- [Use GPG and Blackbox to encrypt files](./gpg/blackbox.md)

‎security/gpg/blackbox.md

+291
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,291 @@
1+
# Use GPG and Blackbox to encrypt files
2+
3+
[Blackbox](https://github.com/StackExchange/blackbox) is a nice way to safely
4+
store secrets needed for deploying applications in a publicly accessible
5+
version control system (like GitHub). The short of it is that it leverages [GPG
6+
encryption](https://www.gnupg.org/) to sign and encrypt files with a given set of keys that can then be
7+
used to decrypt the files when needed on a remote host. After it is set up and your
8+
key is added to the keychain, you shouldn't really need to touch much of
9+
anything except when you want to modify the files that are being encrypted.
10+
11+
## Initial setup
12+
13+
If you already have a GPG key, you can skip ahead to [installing blackbox](#install-blackbox); if you already have blackbox configured on your machine, go ahead and skip to [project setup](#project-setup).
14+
15+
### Create a GPG key
16+
17+
For Mac OS X users, [this
18+
guide should help you install GPG](http://notes.jerzygangi.com/the-best-pgp-tutorial-for-mac-os-x-ever/)
19+
using the program GPG Suite. You can also use [this guide](http://keyring.debian.org/creating-key.html)
20+
to install GPG on the command line.
21+
22+
If at any point in this process you're prompted to create entropy so that the random
23+
number generator can generate a lot of random bytes, a great command to run is:
24+
25+
```bash
26+
sudo dd if=/dev/sda of=/dev/null
27+
```
28+
29+
This just tells your computer to create a copy your main disk and place it
30+
into the void, generating a ton of entropy in the process.
31+
32+
Here's what the output should look like using the command line:
33+
34+
```bash
35+
gpg --gen-key
36+
gpg (GnuPG) 1.4.18; Copyright (C) 2014 Free Software Foundation, Inc.
37+
This is free software: you are free to change and redistribute it.
38+
There is NO WARRANTY, to the extent permitted by law.
39+
40+
gpg: keyring `./secring.gpg' created
41+
gpg: keyring `./pubring.gpg' created
42+
Please select what kind of key you want:
43+
(1) RSA and RSA (default)
44+
(2) DSA and Elgamal
45+
(3) DSA (sign only)
46+
(4) RSA (sign only)
47+
Your selection? 1
48+
RSA keys may be between 1024 and 4096 bits long.
49+
What keysize do you want? (2048) 4096
50+
Requested keysize is 4096 bits
51+
Please specify how long the key should be valid.
52+
0 = key does not expire
53+
<n> = key expires in n days
54+
<n>w = key expires in n weeks
55+
<n>m = key expires in n months
56+
<n>y = key expires in n years
57+
Key is valid for? (0)
58+
Key does not expire at all
59+
Is this correct? (y/N) y
60+
61+
You need a user ID to identify your key; the software constructs the user ID
62+
from the Real Name, Comment and Email Address in this form:
63+
"Heinrich Heine (Der Dichter) <[email protected]>"
64+
Real name: Example key
65+
Email address: [email protected]
66+
Comment:
67+
You selected this USER-ID:
68+
"Example key <[email protected]>"
69+
70+
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
71+
You need a Passphrase to protect your secret key.
72+
73+
We need to generate a lot of random bytes. It is a good idea to perform
74+
some other action (type on the keyboard, move the mouse, utilize the
75+
disks) during the prime generation; this gives the random number
76+
generator a better chance to gain enough entropy.
77+
.......+++++
78+
.....................+++++
79+
......................+++++
80+
+++++
81+
gpg: ./trustdb.gpg: trustdb created
82+
gpg: key B5C7EFB7 marked as ultimately trusted
83+
public and secret key created and signed.
84+
85+
gpg: checking the trustdb
86+
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
87+
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
88+
pub 4096R/B5C7EFB7 2016-03-07
89+
Key fingerprint = 2C75 37E3 044A BE39 6C12 A57F E888 D74A B5C7 EFB7
90+
uid Example key <[email protected]>
91+
sub 4096R/5EC12084 2016-03-07
92+
```
93+
94+
Congratulations! You now have a GPG key.
95+
96+
### Install Blackbox
97+
98+
MacOS users can install Blackbox with Homebrew:
99+
100+
``` bash
101+
brew install blackbox
102+
```
103+
104+
Blackbox can also be installed easily from source:
105+
106+
```bash
107+
# Grab the Blackbox repo with git
108+
git clone https://github.com/StackExchange/blackbox
109+
110+
# Build Blackbox
111+
cd blackbox
112+
sudo make symlinks-install
113+
```
114+
115+
Blackbox has [detailed installation
116+
instructions](https://github.com/StackExchange/blackbox#installation-instructions)
117+
if you need to troubleshoot the installation.
118+
119+
## Project setup
120+
121+
### Initialize Blackbox
122+
123+
The first person to start a project gets the honor of initializing it and being
124+
the first person in the keyring that can decrypt files and add others to the
125+
keyring.
126+
127+
#### GPG history break
128+
129+
Before you initialize Blackbox, it's important to note that there are some
130+
differences between how GPG 1.x and GPG 2.x create encryption artifacts. This
131+
will matter if you're sharing files with someone using a different version: If you
132+
initialize Blackbox using GPG 2.x locally, users of GPG 1.x will fail to decrypt
133+
your files with an error like this:
134+
135+
```
136+
gpg: [don't know]: invalid packet (ctb=00)
137+
gpg: key export failed: invalid packet
138+
```
139+
140+
So, make note of your GPG version now.
141+
142+
```
143+
gpg --version
144+
gpg (GnuPG) 2.2.16
145+
...
146+
```
147+
148+
#### Back to the tutorial...
149+
150+
Once a project is setup to use git, you can do that like so:
151+
152+
``` bash
153+
cd path/to/project
154+
blackbox_initialize
155+
```
156+
157+
That will generate some output that looks like this:
158+
159+
```
160+
Enable blackbox for this git repo? (yes/no) yes
161+
VCS_TYPE: git
162+
163+
164+
NEXT STEP: You need to manually check these in:
165+
git commit -m'INITIALIZE BLACKBOX' keyrings /home/eric/code/blackbox-test/.gitignore
166+
```
167+
168+
Do as it says and commit that change. Next, add yourself as an admin to the project:
169+
170+
```
171+
# Replace my email address with whatever the email address was that you used to
172+
# create your GPG key
173+
blackbox_addadmin [email protected]
174+
```
175+
176+
The output from that command should look something like:
177+
178+
```
179+
gpg: keyring `/home/eric/code/blackbox-test/keyrings/live/secring.gpg' created
180+
gpg: keyring `/home/eric/code/blackbox-test/keyrings/live/pubring.gpg' created
181+
gpg: /home/eric/code/blackbox-test/keyrings/live/trustdb.gpg: trustdb created
182+
gpg: key 25E7098A: public key "Eric van Zanten <[email protected]>" imported
183+
gpg: Total number processed: 1
184+
gpg: imported: 1 (RSA: 1)
185+
186+
187+
NEXT STEP: You need to manually check these in:
188+
git commit -m'NEW ADMIN: [email protected]' keyrings/live/pubring.gpg keyrings/live/trustdb.gpg keyrings/live/blackbox-admins.txt
189+
190+
```
191+
192+
Again, go ahead and commit those changes and push the changes to GitHub. At
193+
this point, you might want to add a file to encrypt. You can do that like so:
194+
195+
``` bash
196+
blackbox_register_new_file Decrypt-me-if-you-can.md
197+
```
198+
199+
That should generate output that looks something like:
200+
201+
```
202+
========== PLAINFILE Decrypt-me-if-you-can.md
203+
========== ENCRYPTED Decrypt-me-if-you-can.md.gpg
204+
========== Importing keychain: START
205+
gpg: Total number processed: 1
206+
gpg: unchanged: 1
207+
========== Importing keychain: DONE
208+
========== Encrypting: Decrypt-me-if-you-can.md
209+
========== Encrypting: DONE
210+
========== Adding file to list.
211+
========== CREATED: Decrypt-me-if-you-can.md.gpg
212+
========== UPDATING REPO:
213+
NOTE: "already tracked!" messages are safe to ignore.
214+
[master eb681eb] registered in blackbox: Decrypt-me-if-you-can.md
215+
2 files changed, 1 insertion(+)
216+
create mode 100644 Decrypt-me-if-you-can.md.gpg
217+
========== UPDATING VCS: DONE
218+
Local repo updated. Please push when ready.
219+
git push
220+
```
221+
222+
You can then push that file to GitHub knowing that only you have the power to
223+
unlock it's secrets.
224+
225+
### Add another user
226+
227+
However, most of the time we'll want to be able to collaborate on things, even
228+
files that have encrypted secrets. To indoctrinate another user into the
229+
project, make sure you've [imported their public
230+
key](https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Step_by_Step_Guide/s1-gnupg-import.html)
231+
and run the following code:
232+
233+
``` bash
234+
git clone [email protected]:datamade/blackbox-test.git
235+
cd blackbox-test
236+
237+
# Replace fake email address with the email address that they registered with
238+
# their GPG key
239+
blackbox_addadmin [email protected]
240+
```
241+
242+
This will generate the same output as before. Before committing the changes, you'll
243+
need to re-encrypt the existing files using the updated keyring in order to allow
244+
the new admin to decrypt them:
245+
246+
``` bash
247+
# Re-encrypt the files
248+
blackbox_update_all_files
249+
250+
# Commit and push the files
251+
git commit -a
252+
git push origin master
253+
```
254+
255+
Now, just to test things out, have the user who was just added pull the changes
256+
and see if they can decrypt the files:
257+
258+
``` bash
259+
cd path/to/project
260+
git pull origin master
261+
blackbox_cat /path/to/encrypted/file
262+
```
263+
264+
That should just `cat` the file out to your terminal. If that worked, you
265+
should also be able to actually edit, add and remove files, too.
266+
267+
### Modifying files
268+
269+
This will launch whatever your default editor is (defined by the $EDITOR
270+
environmental variable), allow you to edit the file and, when you save and
271+
close it, re-encrypt the file all in one go.
272+
273+
``` bash
274+
blackbox_edit /path/to/encrypted/file
275+
```
276+
277+
If that didn't work or it launched some editor that you're not used to, you can
278+
also take in one step at a time like so:
279+
280+
``` bash
281+
blackbox_edit_start /path/to/encrypted/file
282+
283+
# Now open the decrypted file in whatever you want, save it and close it.
284+
285+
blackbox_edit_end /path/to/encrypted/file
286+
287+
```
288+
289+
To add a file, use `blackbox_register_new_file` and to remove a file use
290+
`blackbox_deregister_file`. More commands and details about how to use Blackbox
291+
can be found in their [GitHub repo](https://github.com/StackExchange/blackbox)

0 commit comments

Comments
 (0)