Skip to content

Commit d127444

Browse files
thierry-f-78wtarreau
thierry-f-78
authored and
wtarreau
committedMay 13, 2019
MINOR/DOC: spoe-server: Add documentation
This is the documentation and examples.
1 parent 2e13d0f commit d127444

6 files changed

+194
-58
lines changed
 

‎README

+43-58
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,18 @@
1-
A Random IP reputation service acting as a Stream Processing Offload Agent
2-
--------------------------------------------------------------------------
1+
Multi script langyage Stream Processing Offload Agent
2+
-----------------------------------------------------
33

4-
This is a very simple service that implement a "random" ip reputation
5-
service. It will return random scores for all checked IP addresses. It only
6-
shows you how to implement a ip reputation service or such kind of services
7-
using the SPOE.
4+
This agent receive SPOP message and process it with script languages. The
5+
language register callback with a message. Each callback receive the list
6+
of arguments with types according with the language capabilities. The
7+
callback write variables which are sent as response when the processing
8+
is done.
9+
10+
11+
Compilation
12+
---------------
13+
14+
Actually, the server support Lua and Python. Type "make" with the options:
15+
USE_LUA=1 and/or USE_PYTHON=1.
816

917

1018
Start the service
@@ -19,70 +27,47 @@ binary:
1927
-d Enable the debug mode
2028
-p <port> Specify the port to listen on (default: 12345)
2129
-n <num-workers> Specify the number of workers (default: 5)
30+
-f <file> Load script according with the supported languages
2231

23-
Note: A worker is a thread.
24-
25-
26-
Configure a SPOE to use the service
27-
---------------------------------------
28-
29-
All information about SPOE configuration can be found in "doc/SPOE.txt". Here is
30-
the configuration template to use for your SPOE:
31-
32-
[ip-reputation]
33-
34-
spoe-agent iprep-agent
35-
messages check-client-ip
36-
37-
option var-prefix iprep
38-
39-
timeout hello 100ms
40-
timeout idle 30s
41-
timeout processing 15ms
42-
43-
use-backend iprep-backend
32+
The file processor is recognized using the extension. .lua or .luac for lua and
33+
.py for python. Start example:
4434

45-
spoe-message check-client-ip
46-
args src
47-
event on-client-session
35+
$> ./spoa -d -f ps_lua.lua
4836

37+
$> ./spoa -d -f ps_pyhton.py
4938

50-
The engine is in the scope "ip-reputation". So to enable it, you must set the
51-
following line in a frontend/listener section:
5239

53-
frontend my-front
54-
...
55-
filter spoe engine ip-reputation config /path/spoe-ip-reputation.conf
56-
....
40+
Configure
41+
-------------
5742

58-
where "/path/spoe-ip-reputation.conf" is the path to your SPOE configuration
59-
file. The engine name is important here, it must be the same than the one used
60-
in the SPOE configuration file.
43+
Sample configuration are join to this server:
6144

62-
IMPORTANT NOTE:
63-
Because we want to send a message on the "on-client-session" event, this
64-
SPOE must be attached to a proxy with the frontend capability. If it is
65-
declared in a backend section, it will have no effet.
45+
spoa-server.conf : The HAProxy configuration file using SPOE server
46+
spoa-server.spoe.conf : The SPOP description file used by HAProxy
47+
ps_lua.lua : Processing Lua example
48+
ps_python.py : Processing Python example
6649

6750

68-
Because, in SPOE configuration file, we declare to use the backend
69-
"iprep-backend" to communicate with the service, you must define it in HAProxy
70-
configuration. For example:
51+
Considerations
52+
------------------
7153

72-
backend iprep-backend
73-
mode tcp
74-
timeout server 1m
75-
server iprep-srv 127.0.0.1:12345 check maxconn 5
54+
This server is a beta version. It works fine, but some improvement will be
55+
welcome:
7656

57+
Main process:
7758

78-
In reply to the "check-client-ip" message, this service will set the variable
79-
"ip_score" for the session, an integer between 0 and 100. If unchanged, the
80-
variable prefix is "iprep". So the full variable name will be
81-
"sess.iprep.ip_score".
59+
* Improve log management: Today the log are sent on stdout.
60+
* Improve process management: The dead process are ignored.
61+
* Implement systemd integration.
62+
* Implement threads: It would be fine to implement thread working. Shared
63+
memory is welcome for managing database connection pool and something like
64+
that.
65+
* Add PHP support and some other languages.
8266

83-
You can use it in ACLs to experiment the SPOE feature. For example:
67+
Python:
8468

85-
tcp-request content reject if { var(sess.iprep.ip_score) -m int lt 20 }
69+
* Improve repporting: Catch python error message and repport it in the right
70+
place. Today the error are dumped on stdout. How using syslog for logging
71+
stack traces ?
8672

87-
With this rule, all IP address with a score lower than 20 will be rejected
88-
(Remember, this score is random).
73+
Maybe some other things...

‎print_r.lua

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
function color(index, str)
2+
return "\x1b[" .. index .. "m" .. str .. "\x1b[00m"
3+
end
4+
5+
function nocolor(index, str)
6+
return str
7+
end
8+
9+
function sp(count)
10+
local spaces = ""
11+
while count > 0 do
12+
spaces = spaces .. " "
13+
count = count - 1
14+
end
15+
return spaces
16+
end
17+
18+
function print_rr(p, indent, c, wr)
19+
local i = 0
20+
local nl = ""
21+
22+
if type(p) == "table" then
23+
wr(c("33", "(table)") .. " " .. c("34", tostring(p)) .. " [")
24+
25+
mt = getmetatable(p)
26+
if mt ~= nil then
27+
wr("\n" .. sp(indent+1) .. c("31", "METATABLE") .. ": ")
28+
print_rr(mt, indent+1, c, wr)
29+
end
30+
31+
for k,v in pairs(p) do
32+
if i > 0 then
33+
nl = "\n"
34+
else
35+
wr("\n")
36+
end
37+
wr(nl .. sp(indent+1))
38+
if type(k) == "number" then
39+
wr(c("32", tostring(k)))
40+
else
41+
wr("\"" .. c("32", tostring(k)) .. "\"")
42+
end
43+
wr(": ")
44+
print_rr(v, indent+1, c, wr)
45+
i = i + 1
46+
end
47+
if i == 0 then
48+
wr(" " .. c("35", "/* empty */") .. " ]")
49+
else
50+
wr("\n" .. sp(indent) .. "]")
51+
end
52+
elseif type(p) == "string" then
53+
wr(c("33", "(string)") .. " \"" .. c("34", p) .. "\"")
54+
else
55+
wr(c("33", "(" .. type(p) .. ")") .. " " .. c("34", tostring(p)))
56+
end
57+
end
58+
59+
function print_r(p, col, wr)
60+
if col == nil then col = true end
61+
if wr == nil then wr = function(msg) io.stdout:write(msg) end end
62+
if col == true then
63+
print_rr(p, 0, color, wr)
64+
else
65+
print_rr(p, 0, nocolor, wr)
66+
end
67+
wr("\n")
68+
end

‎ps_lua.lua

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
require("print_r")
2+
3+
print_r("Load lua message processors")
4+
5+
spoa.register_message("check-client-ip", function(args)
6+
print_r(args)
7+
spoa.set_var_null("null", spoa.scope.txn)
8+
spoa.set_var_boolean("boolean", spoa.scope.txn, true)
9+
spoa.set_var_int32("int32", spoa.scope.txn, 1234)
10+
spoa.set_var_uint32("uint32", spoa.scope.txn, 1234)
11+
spoa.set_var_int64("int64", spoa.scope.txn, 1234)
12+
spoa.set_var_uint64("uint64", spoa.scope.txn, 1234)
13+
spoa.set_var_ipv4("ipv4", spoa.scope.txn, "127.0.0.1")
14+
spoa.set_var_ipv6("ipv6", spoa.scope.txn, "1::f")
15+
spoa.set_var_str("str", spoa.scope.txn, "1::f")
16+
spoa.set_var_bin("bin", spoa.scope.txn, "1::f")
17+
end)

‎ps_python.py

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
from pprint import pprint
2+
import spoa
3+
import ipaddress
4+
5+
def check_client_ip(args):
6+
pprint(args)
7+
spoa.set_var_null("null", spoa.scope_txn)
8+
spoa.set_var_boolean("boolean", spoa.scope_txn, True)
9+
spoa.set_var_int32("int32", spoa.scope_txn, 1234)
10+
spoa.set_var_uint32("uint32", spoa.scope_txn, 1234)
11+
spoa.set_var_int64("int64", spoa.scope_txn, 1234)
12+
spoa.set_var_uint64("uint64", spoa.scope_txn, 1234)
13+
spoa.set_var_ipv4("ipv4", spoa.scope_txn, ipaddress.IPv4Address(u"127.0.0.1"))
14+
spoa.set_var_ipv6("ipv6", spoa.scope_txn, ipaddress.IPv6Address(u"1::f"))
15+
spoa.set_var_str("str", spoa.scope_txn, "1::f")
16+
spoa.set_var_bin("bin", spoa.scope_txn, "1:\x01:\x02f\x00\x00")
17+
return
18+
19+
20+
spoa.register_message("check-client-ip", check_client_ip)

‎spoa-server.conf

+33
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
global
2+
debug
3+
4+
defaults
5+
mode http
6+
option httplog
7+
option dontlognull
8+
timeout connect 5000
9+
timeout client 5000
10+
timeout server 5000
11+
12+
listen test
13+
mode http
14+
bind :10001
15+
filter spoe engine spoa-server config spoa-server.spoe.conf
16+
http-request set-var(req.a) var(txn.iprep.null),debug
17+
http-request set-var(req.a) var(txn.iprep.boolean),debug
18+
http-request set-var(req.a) var(txn.iprep.int32),debug
19+
http-request set-var(req.a) var(txn.iprep.uint32),debug
20+
http-request set-var(req.a) var(txn.iprep.int64),debug
21+
http-request set-var(req.a) var(txn.iprep.uint64),debug
22+
http-request set-var(req.a) var(txn.iprep.ipv4),debug
23+
http-request set-var(req.a) var(txn.iprep.ipv6),debug
24+
http-request set-var(req.a) var(txn.iprep.str),debug
25+
http-request set-var(req.a) var(txn.iprep.bin),debug
26+
http-request redirect location /%[var(sess.iprep.ip_score)]
27+
28+
backend spoe-server
29+
mode tcp
30+
balance roundrobin
31+
timeout connect 5s
32+
timeout server 3m
33+
server spoe-server 127.0.0.1:12345

‎spoa-server.spoe.conf

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
[spoa-server]
2+
3+
spoe-agent spoa-server
4+
messages check-client-ip
5+
option var-prefix iprep
6+
timeout hello 100ms
7+
timeout idle 30s
8+
timeout processing 15ms
9+
use-backend spoe-server
10+
11+
spoe-message check-client-ip
12+
args always_true int(1234) src ipv6(::55) req.fhdr(host)
13+
event on-frontend-http-request

0 commit comments

Comments
 (0)
Please sign in to comment.