Skip to content

Commit 2935e94

Browse files
phusluphuslu
committed
update readme
1 parent d3e1285 commit 2935e94

File tree

1 file changed

+8
-272
lines changed

1 file changed

+8
-272
lines changed

Diff for: ‎README.md

+8-272
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,31 @@
11
# nginx-ssl-fingerprint
22

3-
A stable nginx module for SSL/TLS ja3 fingerprint, with high [performance](#performance).
3+
A high performance nginx module for ja3 and http2 fingerprint.
44

55
## Patches
6-
- [nginx - save client hello fingerprint](patches/nginx.patch)
7-
- [openssl - expose client hello data](patches/openssl.1_1_1.patch)
6+
- [nginx - save ja3/http2 fingerprint](patches/nginx.patch)
7+
- [openssl - expose clienthello data](patches/openssl.1_1_1.patch)
88

99
## Configuration
1010

1111
### Variables
1212

1313
| Name | Default Value | Comments |
1414
| ----------------- | ------------- | ----------------------------------------------------------- |
15-
| http_ssl_greased | 0 | Chrome grease flag |
15+
| http_ssl_greased | 0 | TLS greased flag. |
1616
| http_ssl_ja3 | NULL | The ja3 fingerprint for a SSL connection for a HTTP server. |
17+
| http2_fingerprint | NULL | The http2 fingerprint. |
1718

1819
#### Example
1920

2021
```nginx
2122
http {
2223
server {
23-
listen 127.0.0.1:8443 ssl;
24+
listen 127.0.0.1:8443 ssl http2;
2425
ssl_certificate cert.pem;
2526
ssl_certificate_key priv.key;
2627
error_log /dev/stderr debug;
27-
return 200 "$http_ssl_ja3";
28+
return 200 "ja3: $http_ssl_ja3\nh2fp: $http2_fingerprint";
2829
}
2930
}
3031
```
@@ -47,276 +48,11 @@ $ patch -p1 -d nginx < nginx-ssl-fingerprint/patches/nginx.patch
4748
# Configure & Build
4849

4950
$ cd nginx
50-
$ ASAN_OPTIONS=symbolize=1 ./auto/configure --with-openssl=$(pwd)/../openssl --add-module=$(pwd)/../nginx-ssl-fingerprint --with-http_ssl_module --with-stream_ssl_module --with-debug --with-stream --with-cc-opt="-fsanitize=address -O -fno-omit-frame-pointer" --with-ld-opt="-L/usr/local/lib -Wl,-E -lasan"
51+
$ ASAN_OPTIONS=symbolize=1 ./auto/configure --with-openssl=$(pwd)/../openssl --add-module=$(pwd)/../nginx-ssl-fingerprint --with-http_ssl_module --with-stream_ssl_module --with-debug --with-stream --with-http_v2_module --with-cc-opt="-fsanitize=address -O -fno-omit-frame-pointer" --with-ld-opt="-L/usr/local/lib -Wl,-E -lasan"
5152
$ make
5253

5354
# Test
5455

5556
$ objs/nginx -p . -c $(pwd)/../nginx-ssl-fingerprint/nginx.conf
5657
$ curl -k https://127.0.0.1:8444
5758
```
58-
59-
## Performance
60-
61-
### Server
62-
63-
| Type | Service | Cores | Memeory(G) |
64-
| ------ | ------------------- | ----- | ---------- |
65-
| Server | nginx with 5 worker | 8 | 8 |
66-
| Client | wrk | 8 | 8 |
67-
68-
### Performance Results
69-
70-
```bash
71-
for i in $(seq 1 10); do
72-
wrk https://localhost/ --latency -t48 -d15 -c2000 >/tmp/wrk.log.$i
73-
done
74-
```
75-
76-
- QPS: Average Req/Second in 10 times
77-
- Latency: Average 50% latency (ms) in 10 times
78-
79-
v0.2.0 ~ v0.3.0
80-
81-
| WRK Connection | QPS Cost | Origin Req/Sec | Origin Latency | Req/Sec with fingerprint | Latency with fingerprint |
82-
| -------------- | -------- | -------------- | -------------- | ------------------------ | ------------------------ |
83-
| 50 | 9.98% | 2990.26 | 14.151 | 2694.96 | 15.957 |
84-
| 100 | 26.2% | 3718.62 | 24.397 | 2743.47 | 32.091 |
85-
| 200 | 20.1% | 3680.09 | 49.332 | 2941.92 | 63.351 |
86-
| 500 | 27.1% | 3503.64 | 111.085 | 2555.59 | 149.989 |
87-
| 1000 | 22.4% | 3333.68 | 134.187 | 2586.89 | 169.36 |
88-
| 1500 | 28.6% | 3497.15 | 135.949 | 2498.74 | 177.784 |
89-
| 2000 | 37.0% | 2390.82 | 229.657 | 1506.78 | 275.601 |
90-
91-
v0.4.0
92-
93-
| WRK Connection | QPS Cost | Origin Req/Sec | Origin Latency | Req/Sec with fingerprint | Latency with fingerprint |
94-
| -------------- | -------- | -------------- | -------------- | ------------------------ | ------------------------ |
95-
| 50 | 8.60% | 36795.9 | 1.211 | 33624.2 | 1.33 |
96-
| 100 | 8.4% | 36831.2 | 2.43545 | 33734.6 | 66.8545 |
97-
| 200 | 8.39% | 36862.5 | 4.814 | 33767.2 | 5.28 |
98-
| 500 | 8.45 | 36598.8 | 11.827 | 33505.5 | 12.786 |
99-
| 1000 | 9.63% | 33657.1 | 20.877 | 36900.1 | 20.059 |
100-
| 1500 | 8.95% | 36806.3 | 27.591 | 33511 | 28.155 |
101-
| 2000 | 8.71% | 37460.2 | 30.664 | 34194.7 | 31.504 |
102-
103-
## TLS Client Hello Packet
104-
```
105-
Frame 103: 561 bytes on wire (4488 bits), 561 bytes captured (4488 bits) on interface utun7, id 0
106-
Null/Loopback
107-
Internet Protocol Version 4, Src: 10.22.76.29, Dst: 143.92.64.129
108-
Transmission Control Protocol, Src Port: 56795, Dst Port: 443, Seq: 1, Ack: 1, Len: 517
109-
Source Port: 56795
110-
Destination Port: 443
111-
[Stream index: 8]
112-
[Conversation completeness: Complete, WITH_DATA (31)]
113-
[TCP Segment Len: 517]
114-
Sequence Number: 1 (relative sequence number)
115-
Sequence Number (raw): 4163556192
116-
[Next Sequence Number: 518 (relative sequence number)]
117-
Acknowledgment Number: 1 (relative ack number)
118-
Acknowledgment number (raw): 1321024790
119-
0101 .... = Header Length: 20 bytes (5)
120-
Flags: 0x018 (PSH, ACK)
121-
Window: 4096
122-
[Calculated window size: 262144]
123-
[Window size scaling factor: 64]
124-
Checksum: 0xc517 [unverified]
125-
[Checksum Status: Unverified]
126-
Urgent Pointer: 0
127-
[Timestamps]
128-
[SEQ/ACK analysis]
129-
TCP payload (517 bytes)
130-
Transport Layer Security
131-
TLSv1.3 Record Layer: Handshake Protocol: Client Hello
132-
Content Type: Handshake (22)
133-
Version: TLS 1.0 (0x0301)
134-
Length: 512
135-
Handshake Protocol: Client Hello
136-
Handshake Type: Client Hello (1)
137-
Length: 508
138-
Version: TLS 1.2 (0x0303)
139-
Random: cba554e69ef810c86d3e843f137dc3759fed3f0d6c621f1fb45acb7f92560b48
140-
Session ID Length: 32
141-
Session ID: b0a3a558861404d133bc11f9d6b4a4b32b60caaa1559e9332165dc2ac1643d28
142-
Cipher Suites Length: 32
143-
Cipher Suites (16 suites)
144-
Cipher Suite: Reserved (GREASE) (0x6a6a)
145-
Cipher Suite: TLS_AES_128_GCM_SHA256 (0x1301)
146-
Cipher Suite: TLS_AES_256_GCM_SHA384 (0x1302)
147-
Cipher Suite: TLS_CHACHA20_POLY1305_SHA256 (0x1303)
148-
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
149-
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
150-
Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (0xc02c)
151-
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
152-
Cipher Suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca9)
153-
Cipher Suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 (0xcca8)
154-
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
155-
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
156-
Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
157-
Cipher Suite: TLS_RSA_WITH_AES_256_GCM_SHA384 (0x009d)
158-
Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
159-
Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
160-
Compression Methods Length: 1
161-
Compression Methods (1 method)
162-
Compression Method: null (0)
163-
Extensions Length: 403
164-
Extension: Reserved (GREASE) (len=0)
165-
Type: Reserved (GREASE) (27242)
166-
Length: 0
167-
Data: <MISSING>
168-
Extension: server_name (len=29)
169-
Type: server_name (0)
170-
Length: 29
171-
Server Name Indication extension
172-
Extension: extended_master_secret (len=0)
173-
Type: extended_master_secret (23)
174-
Length: 0
175-
Extension: renegotiation_info (len=1)
176-
Type: renegotiation_info (65281)
177-
Length: 1
178-
Renegotiation Info extension
179-
Extension: supported_groups (len=10)
180-
Type: supported_groups (10)
181-
Length: 10
182-
Supported Groups List Length: 8
183-
Supported Groups (4 groups)
184-
Extension: ec_point_formats (len=2)
185-
Type: ec_point_formats (11)
186-
Length: 2
187-
EC point formats Length: 1
188-
Elliptic curves point formats (1)
189-
Extension: session_ticket (len=0)
190-
Type: session_ticket (35)
191-
Length: 0
192-
Data (0 bytes)
193-
Extension: application_layer_protocol_negotiation (len=14)
194-
Type: application_layer_protocol_negotiation (16)
195-
Length: 14
196-
ALPN Extension Length: 12
197-
ALPN Protocol
198-
Extension: status_request (len=5)
199-
Type: status_request (5)
200-
Length: 5
201-
Certificate Status Type: OCSP (1)
202-
Responder ID list Length: 0
203-
Request Extensions Length: 0
204-
Extension: signature_algorithms (len=18)
205-
Type: signature_algorithms (13)
206-
Length: 18
207-
Signature Hash Algorithms Length: 16
208-
Signature Hash Algorithms (8 algorithms)
209-
Extension: signed_certificate_timestamp (len=0)
210-
Type: signed_certificate_timestamp (18)
211-
Length: 0
212-
Extension: key_share (len=43)
213-
Type: key_share (51)
214-
Length: 43
215-
Key Share extension
216-
Extension: psk_key_exchange_modes (len=2)
217-
Type: psk_key_exchange_modes (45)
218-
Length: 2
219-
PSK Key Exchange Modes Length: 1
220-
PSK Key Exchange Mode: PSK with (EC)DHE key establishment (psk_dhe_ke) (1)
221-
Extension: supported_versions (len=7)
222-
Type: supported_versions (43)
223-
Length: 7
224-
Supported Versions length: 6
225-
Supported Version: Reserved (GREASE) (0x7a7a)
226-
Supported Version: TLS 1.3 (0x0304)
227-
Supported Version: TLS 1.2 (0x0303)
228-
Extension: compress_certificate (len=3)
229-
Type: compress_certificate (27)
230-
Length: 3
231-
Algorithms Length: 2
232-
Algorithm: brotli (2)
233-
Extension: application_settings (len=5)
234-
Type: application_settings (17513)
235-
Length: 5
236-
ALPS Extension Length: 3
237-
Supported ALPN List
238-
Supported ALPN Length: 2
239-
Supported ALPN: h2
240-
Extension: Reserved (GREASE) (len=1)
241-
Type: Reserved (GREASE) (64250)
242-
Length: 1
243-
Data: 00
244-
Extension: padding (len=191)
245-
Type: padding (21)
246-
Length: 191
247-
Padding Data: 000000000000000000000000000000000000000000000000000000000000000000000000…
248-
[JA3 Fullstring: 771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21,29-23-24,0]
249-
[JA3: cd08e31494f9531f560d64c695473da9]
250-
251-
```
252-
253-
### Ja3
254-
255-
As we know that, we canJA3 is a method for creating SSL/TLS client fingerprints that should be easy to produce on any platform and can be easily shared for threat intelligence.
256-
257-
This module adds new nginx variables for the SSL/TLS ja3 fingerprint. For more information, please see the [salesforce ja3](https://github.com/salesforce/ja3)
258-
259-
JA3 take 5 fields from TLS Client Hello packet ([example client-hello](./clien-hello.pcap)), each filed convert to decimal and join by '-', then the 5 fields string join by  ','   and then get the ja3 full string.
260-
261-
The ja3 full string's length is not fixed, it's hard to compare. So finally hash the ja3 full string to md5 for easy store or compare.
262-
263-
One Ja3 full string contains 5 parts:
264-
265-
- `TLSVersion`: convert the hex to decimal
266-
- `Ciphers`: convert the hex to decimal and join by '-'
267-
- `Extensions`: use type value and join by '-' (it's decimal, no need to convert)
268-
- `Supported Groups/Elliptic_Curves`: convert hex to decimal and join by '-' (exclude Reserved GREASE )
269-
- `EC_Point_Formats`: always be 0 or empty (needs to confirm more details)
270-
271-
### Field 1 - TLSVersion
272-
273-
• TLS Record - Version: minimum supported TLS version (in TLS 1.2 and before). In TLS 1.3, this field is not really used and MUST be 0x0303 ("TLS 1.2") or 0x301 ("TLS 1.0") for compatibility purposes. Reference: RFC 8446 (page 79)
274-
275-
• Client Hello - Version: maximum supported TLS version (in TLS 1.2 and before). In TLS 1.3, this field is not used but MUST be set to 0x0303 ("TLS 1.2"). Reference: RFC 8446 (4.1.2. Client Hello)
276-
277-
Currently, the nginx-ssl-fingerprint used nginx connection, it means the tls version is negotiated.
278-
279-
We are trying to use another TLSVersion which donot take affect by server side.
280-
281-
### Filed 2 - Ciphers
282-
283-
The filed does not special.
284-
For the new OpenSSL lib, the support ciphers count can reach 30 or higher, such as curl.
285-
286-
### Filed 3 - Extensions
287-
288-
The extensions order is random, but relay on the OpenSSL library.  Every tool has its own SSL lib, so even if restarting the devices, the TLS fingerprint does not change.
289-
290-
|OpenSSL ID | Extension name| Remark|
291-
|------ | ------ | ------ |
292-
|0| TLSEXT_TYPE_server_name| |
293-
|5| TLSEXT_TYPE_status_request |patched after v0.3.0|
294-
|10| TLSEXT_TYPE_supported_groups||
295-
|11| TLSEXT_TYPE_ec_point_formats||
296-
|13| TLSEXT_TYPE_signature_algorithms | |
297-
|16| TLSEXT_TYPE_application_layer_protocol_negotiation ||
298-
|18| TLSEXT_TYPE_signed_certificate_timestamp||
299-
|21|TLSEXT_TYPE_padding||
300-
|23| TLSEXT_TYPE_extended_master_secret| |
301-
|27|TLSEXT_TYPE_compress_certificate|patched|
302-
|28|TLSEXT_TYPE_record_size_limit|pathed|
303-
|35| TLSEXT_TYPE_session_ticket ||
304-
|43| TLSEXT_TYPE_supported_versions||
305-
|45|TLSEXT_TYPE_psk_kex_modes||
306-
|51| TLSEXT_TYPE_key_share||
307-
|17513|TLSEXT_TYPE_application_settings|patched after v0.3.0|
308-
|0xff01|TLSEXT_TYPE_renegotiate| |
309-
|-|Reserved||Ignore this extension|
310-
311-
312-
`TLSEXT_TYPE_supported_groups` and `TLSEXT_TYPE_elliptic_curves` they are same filed. More details refer:
313-
https://github.com/openssl/openssl/blob/master/include/openssl/tls1.h#L102-L103
314-
315-
316-
### Filed 4 - EllipticCurves
317-
318-
The filed use extensions `TLSEXT_TYPE_supported_groups`(`TLSEXT_TYPE_elliptic_curves`) supported group value, and convert to integer.
319-
320-
### Filed 5 - EllipticCurvePointFormats
321-
322-
The filed use extensions `TLSEXT_TYPE_ec_point_formats` value, and convert to int.

0 commit comments

Comments
 (0)