Skip to content

Commit b979544

Browse files
committed
Merge branch 'release-v20210714' into 'master'
release v20210714 See merge request knot/shotgun!54
2 parents 09e4a6b + 8efbce4 commit b979544

File tree

9 files changed

+125
-51
lines changed

9 files changed

+125
-51
lines changed

Dockerfile

+5-14
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,26 @@
11
FROM ubuntu:focal
22
ENV DEBIAN_FRONTEND=noninteractive
33
RUN \
4+
apt-get update -qq && \
5+
apt-get install -y -qqq software-properties-common && \
6+
add-apt-repository -y ppa:dns-oarc/dnsjit && \
47
apt-get update -qq && \
58
apt-get install -y -qqq \
9+
dnsjit \
10+
dnsjit-dev \
611
python3 \
712
python3-pip \
813
tshark \
914
jq \
10-
libck-dev \
11-
libluajit-5.1-dev \
12-
libpcap-dev \
13-
liblmdb-dev \
14-
libgnutls28-dev \
1515
libnghttp2-dev \
1616
luajit \
1717
libuv1-dev \
18-
libgoogle-perftools-dev \
1918
make \
2019
automake \
2120
libtool \
2221
pkg-config \
2322
git && \
2423
rm -rf /var/lib/apt/lists/*
25-
RUN \
26-
git clone https://github.com/DNS-OARC/dnsjit.git && \
27-
cd dnsjit && \
28-
./autogen.sh && \
29-
./configure && \
30-
make && \
31-
make install && \
32-
cd ..
3324

3425
COPY . /shotgun
3526
WORKDIR /shotgun

NEWS

+66
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
DNS Shotgun v20210714
2+
=====================
3+
4+
Incompatible changes
5+
--------------------
6+
- dnsjit 1.2.0+ is now required
7+
- dnssim component was moved from dnsjit to this repository and has to be
8+
compiled separately (!52)
9+
10+
Improvements
11+
------------
12+
- replay: new option `stop_after_s` added to input (!40)
13+
- tools/plot-packat-rate: new tool to plot packet rate in PCAPs (!46)
14+
- replay/shotgun: support reading PCAP from stdin (!36)
15+
- pcap/merge-chunks: utility to merge chunks on-the-fly (!37)
16+
- tools/plot-response-rate: add `--rcodes-above-pct` option (!43)
17+
- tools/plot-response-rate: add `--skip-total` option (!47)
18+
- tools/plot-response-rate: add `--sum-rcodes` option (!47)
19+
- tools/plot-latency: add `--group` option to plot aggregate data (!53)
20+
- pcap/cut-pcap: new script to effetively trim a sorted PCAP (!50)
21+
- configs: plot response rate chart with non-zero rcodes by default (!44)
22+
- pcap/extract-clients: add --seed option for deterministic splitting (!34)
23+
- pcap/filter-dnsq: improved malformed packet handling (!31)
24+
- pcap/extract-clients: detect non-monotonic timestamps (!31)
25+
- replay/shotgun: use faster mmpcap (!36)
26+
- tools/plot-response-rate: add markers for plotting rcodes (!41)
27+
- tools/plot-response-rate: improve color selection (!41, !42)
28+
29+
Bugfixes
30+
--------
31+
- pcap/extract-clients: fix imprecise timestamps and off-by-one error (!32)
32+
- tools/plot-client-distribution: fix handling of empty query count intervals (!38)
33+
- replay: add workaround to support /32 and /128 addresses (!39)
34+
35+
36+
DNS Shotgun v20210203
37+
=====================
38+
39+
- new user interface
40+
- replay.py is now the main tool instead of shotgun.lua
41+
- new configuration format to support complex scenarios
42+
- mixed-protocol traffic replay is supported
43+
- some scripts in `tools/` were renamed
44+
- fast scripts for processing PCAP data
45+
- can be found in `pcap/`
46+
- these replaced the now removed pellet.py and scale.py
47+
- added IPv4 support
48+
- DoH is now fully supported
49+
- comprehensive usage documentation is available at https://dns-shotgun.readthedocs.io
50+
- minimum required dnsjit version is 1.1.0
51+
52+
53+
DNS Shotgun v20200914
54+
=====================
55+
56+
- fully supported UDP, TCP and DNS-over-TLS with
57+
[dnsjit](https://github.com/DNS-OARC/dnsjit) 1.0.0
58+
- fully supported DNS-over-HTTPS with development version of dnsjit
59+
- traffic can be replayed only over IPv6
60+
- user interface
61+
- may be unstable
62+
- only very basic UI available
63+
- more complex scenarios are no supported yet
64+
(e.g. simultaneously using multiple protocols)
65+
- pellet.py is functional, but it is very slow and requires python-dpkt from
66+
master

docs/installation.md

+38-28
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,16 @@
11
# Installation
22

3-
There are two options for using DNS Shotgun. You can either install the
4-
dependencies and use the scripts from the repository directly, or use a
5-
pre-built docker image.
6-
7-
## Using script directly
8-
9-
You can use the toolchain scripts directly from the git repository. You need to
10-
ensure you have the required dependencies installed. Also make sure to check
11-
out some tagged version, as the development happens in master branch.
12-
13-
```
14-
$ git clone https://gitlab.nic.cz/knot/shotgun.git
15-
$ git checkout v20210203
16-
```
17-
18-
### Dependencies
19-
20-
When using the scripts directly, the following dependencies are needed. If you
21-
only wish to process shotgun JSON output (e.g. plot charts), then dnsjit isn't
22-
required.
23-
24-
- [dnsjit](https://github.com/DNS-OARC/dnsjit): Can be installed from [DNS-OARC
25-
repositories](https://dev.dns-oarc.net/packages/).
26-
- Python 3.6 or later
27-
- Python dependencies from [requirements.txt](https://gitlab.nic.cz/knot/shotgun/-/blob/master/requirements.txt)
28-
- (optional) tshark/wireshark for some PCAP pre-processing
3+
There are two options for using DNS Shotgun. You can either use a pre-built
4+
docker image, or install the dependencies, compile the dnssim module and use
5+
the scripts from the repository directly.
296

307
## Docker Image
318

329
Pre-built image can be obtained from [CZ.NIC DNS Shotgun
3310
Registry](https://gitlab.nic.cz/knot/shotgun/container_registry/65).
3411

3512
```
36-
$ docker pull registry.nic.cz/knot/shotgun:v20210203
13+
$ docker pull registry.nic.cz/knot/shotgun:v20210714
3714
```
3815

3916
Alternately, you can build the image yourself from Dockerfile in the repository.
@@ -48,6 +25,39 @@ Alternately, you can build the image yourself from Dockerfile in the repository.
4825
$ docker run \
4926
--network host \
5027
-v "$PWD:/mnt" \
51-
registry.nic.cz/knot/shotgun:v20210203 \
28+
registry.nic.cz/knot/shotgun:v20210714 \
5229
$COMMAND
5330
```
31+
32+
## Using scripts from sources
33+
34+
You can use the toolchain scripts directly from the git repository. You need to
35+
ensure you have the required dependencies installed and the compile and install
36+
the dnssim module. Also make sure to check out some tagged version, as the
37+
development happens in master branch.
38+
39+
```
40+
$ git clone https://gitlab.nic.cz/knot/shotgun.git
41+
$ git checkout v20210714
42+
$ cd shotgun/replay/dnssim
43+
$ ./autogen.sh
44+
$ ./configure
45+
$ make
46+
$ make install
47+
```
48+
49+
### Dependencies
50+
51+
When using the scripts directly, the following dependencies are needed.
52+
53+
If you only wish to process shotgun JSON output (e.g. plot charts), then dnsjit
54+
and compiling the dnssim module isn't required.
55+
56+
- [dnsjit 1.2+](https://github.com/DNS-OARC/dnsjit): Can be installed from [DNS-OARC
57+
repositories](https://dev.dns-oarc.net/packages/).
58+
- libuv
59+
- libnghttp2
60+
61+
- Python 3.6 or later
62+
- Python dependencies from [requirements.txt](https://gitlab.nic.cz/knot/shotgun/-/blob/master/requirements.txt)
63+
- (optional) tshark/wireshark for some PCAP pre-processing
File renamed without changes.

replay/dnssim/CHANGELOG.md

+5
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
dnssim v20210714
2+
================
3+
- dnssim was moved from dnsjit to shotgun, this implies new
4+
lua name: "shotgun.output.dnssim"
5+
16
dnssim v20210129
27
================
38

replay/dnssim/configure.ac

+5-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
AC_PREREQ(2.61)
2-
AC_INIT([shotgun-output-dnssim], [20210129],,)
2+
AC_INIT([shotgun-output-dnssim], [20210714],,)
33
AM_INIT_AUTOMAKE([-Wall -Werror foreign subdir-objects])
44
AC_CONFIG_HEADER([src/config.h])
55

@@ -21,6 +21,10 @@ PKG_CHECK_MODULES([libuv], [libuv])
2121
PKG_CHECK_MODULES([libnghttp2], [libnghttp2])
2222
AC_CHECK_LIB([gnutls], [gnutls_init],, [AC_MSG_ERROR([libgnutls not found])])
2323

24+
# Checks for support.
25+
AC_SEARCH_LIBS([clock_gettime])
26+
AC_CHECK_FUNCS([clock_nanosleep])
27+
2428
# Output Makefiles
2529
AC_CONFIG_FILES([
2630
Makefile

replay/dnssim/src/output/dnssim.c

+3-2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,8 @@
22
* SPDX-License-Identifier: GPL-3.0-or-later
33
*/
44

5+
#include "config.h"
6+
57
#include "output/dnssim.h"
68
#include "output/dnssim/internal.h"
79
#include "output/dnssim/ll.h"
@@ -28,8 +30,7 @@ static uint64_t _now_ms()
2830
now_ms += ts.tv_nsec / 1000000;
2931
return now_ms;
3032
#else
31-
mlfatal("clock_gettime() not available");
32-
return 0;
33+
#error "clock_gettime() is not available"
3334
#endif
3435
}
3536

replay/dnssim/src/output/dnssim.lua

+1-1
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@ local C = ffi.C
149149

150150
local DnsSim = {}
151151

152-
local _DNSSIM_VERSION = 20210129
152+
local _DNSSIM_VERSION = 20210714
153153
local _DNSSIM_JSON_VERSION = 20200527
154154

155155
-- Create a new DnsSim output for up to max_clients.

replay/shotgun.lua

+2-5
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ local object = require("dnsjit.core.objects")
44
local log = require("dnsjit.core.log")
55
local dnssim = require("shotgun.output.dnssim")
66

7-
local DNSSIM_REQ_VERSION = 20210129
7+
local DNSSIM_REQ_VERSION = 20210714
88
local has_check_version, version = pcall(dnssim.check_version, DNSSIM_REQ_VERSION)
99
if not has_check_version or version == nil then
1010
log.fatal(string.format(
@@ -45,7 +45,7 @@ local function send_thread_main(thr)
4545

4646
-- output must be global (per thread) to be accesible in loadstring()
4747
-- luacheck: globals output, ignore log
48-
output = require("dnsjit.output.dnssim").new(thr:pop())
48+
output = require("shotgun.output.dnssim").new(thr:pop())
4949
local log = output:log(thr:pop())
5050

5151
output:target(thr:pop(), thr:pop())
@@ -65,9 +65,6 @@ local function send_thread_main(thr)
6565
elseif protocol_func == "tls" then
6666
cmd = cmd .. "('" .. gnutls_priority .. "')"
6767
elseif protocol_func == "https2" then
68-
if type(output.https2) ~= "function" then
69-
log:fatal("https2 isn't supported with this version of dnsjit")
70-
end
7168
cmd = cmd .. "({ method = '" .. http_method .. "' }, '" .. gnutls_priority .. "')"
7269
else
7370
log:fatal("unknown protocol_func: " .. protocol_func)

0 commit comments

Comments
 (0)