You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Fix structlog dependency for app (#280)
* zipfile fix (#284)
* Fix bug 286 random token replacement (#287)
* Fix bug 286 random token replacement
* Change perdayvolume generator logic to get random token value replacement
* Versioning scheme (#278)
* [global] perDayVolume (#288)
* exclude global from perDayVolume assignment
* Address comment
* Fix security vulnerability issue (#289)
* Fix custom plugin stale docs (#290)
* Server fix (#293)
* Flag added
* server fix for count and env clean
* Fix bug 285 (#297)
* Add syslogAddHeader config directive (#296)
* Add syslog header to event in syslog mode
* timezone setting bugfix #249
* Using multiprocess pool to address the OOM issue (#301)
* Using multiprocess pool to address the OOM issue
* Fix test case fail
* Remove workerQueue unfinished tasks (#302)
* Bumped version to 6.5.2
Copy file name to clipboardexpand all lines: docs/CONTRIBUTE_CODE.md
+13-3
Original file line number
Diff line number
Diff line change
@@ -5,18 +5,26 @@ If you want to contribute code to eventgen, please read over the following guide
5
5
6
6
## Pull request guidelines
7
7
8
-
9
8
If you want to contribute to an eventgen repo, please use a GitHub pull request. This is the fastest way for us to evaluate your code and to merge it into the code base. Please don’t file an issue with snippets of code. Doing so means that we need to manually merge the changes in and update any appropriate tests. That decreases the likelihood that your code is going to get included in a timely manner. Please use pull requests.
10
9
10
+
11
+
## Release versioning guidelines
12
+
13
+
Major Release — Increment the first digit by 1 if the new features break backwards compatibility/current features
14
+
15
+
Minor Release — Increment the middle digit by 1 if the new features don’t break any existing features and are compatible with the app in it’s current state
16
+
17
+
Patch Release — Increment the last digit by 1 if you’re publishing bug/patch fixes to your app
18
+
11
19
### Get started
12
20
13
21
If you’d like to work on a pull request and you’ve never submitted code before, follow these steps:
14
22
1. fork eventgen to your github workspace
15
23
2. If you want to fix bugs or make enhancement, please make sure there is a issue in eventgen project. Refer [this guide](FILE_ISSUES.md) to create a issue.
16
24
17
-
18
25
After that, you’re ready to start working on code.
19
26
27
+
20
28
### Working on the code
21
29
22
30
The process of submitting a pull request is fairly straightforward and generally follows the same pattern each time:
@@ -75,6 +83,7 @@ The message summary should be a one-sentence description of the change, and it m
75
83
76
84
**Note**: please squash you changes in one commit before firing the pull request. One commit in one PR keeps the git history clean.
77
85
86
+
78
87
#### Step 3: Rebase onto upstream
79
88
80
89
Before you send the pull request, be sure to rebase onto the upstream source. This ensures your code is running on the latest available code. We prefer rebase instead of merge when upstream changes. Rebase keeps the git history clearer.
@@ -83,6 +92,7 @@ git fetch upstream
83
92
git rebase upstream/master
84
93
```
85
94
95
+
86
96
#### Step 4: Run the tests
87
97
88
98
The is a place holder as well. We should write about
@@ -101,6 +111,7 @@ Next, push your changes to your clone:
101
111
git push origin fix/issue123
102
112
```
103
113
114
+
104
115
#### Step 6: Submit the pull request
105
116
106
117
Before creating a pull request, here are some recommended **check points**.
@@ -118,7 +129,6 @@ Next, create a pull request from your branch to the eventgen develop branch.
118
129
Mark @lephino , @arctan5x , @jmeixensperger , @li-wu , @GordonWang as the reviewers.
119
130
120
131
121
-
122
132
## Code style and formatting tools
123
133
124
134
Since Eventgen is written in python, we apply a coding style based on [PEP8](https://www.python.org/dev/peps/pep-0008/).
First, we import the OutputPlugin superclass. For output plugins, they define a constant MAXQUEUELENGTH to determine the maximum amount of items in queue before forcing a queue flush.
47
+
First, we import the OutputPlugin superclass. For output plugins, they define a constant `MAXQUEUELENGTH` to determine the maximum amount of items in queue before forcing a queue flush.
48
+
49
+
`useOutputQueue` is set to `True` here to use the output queue which functions as a reduce step when you need to maintain a single thread or a limited number of threads outputting data
42
50
43
51
``__init__()`` is very simple. It calls its superclass init and sets one variable, firsttime. ``flush()`` is also very simple.
44
52
If it's the first time, open the file /dev/null, otherwise, output the queue by writing it to the already open file.
@@ -56,34 +64,34 @@ class SplunkStreamOutputPlugin(OutputPlugin):
56
64
intSettings = [ 'splunkPort' ]
57
65
```
58
66
59
-
MAXQUEUELENGTH should look normal, but these other class variables need a little explanation.
67
+
`MAXQUEUELENGTH` should look normal, but these other class variables need a little explanation.
60
68
61
69
### Configuration Validation
62
70
Config validation is a modular system in Eventgen, and plugins must be allowed to specify additional configuration parameters that the main Eventgen will consider valid and store.
63
-
*Note that eventgen.conf.spec generation is not yet automated, which means plugins must ship with the default distribution and eventgen.conf.spec must be maintained manually.*
71
+
> Note that `eventgen.conf.spec` generation is not yet automated, which means plugins must ship with the default distribution and eventgen.conf.spec must be maintained manually.
64
72
Eventually spec file generation will be automated as well.
65
73
66
74
The main configuration of Eventgen validates itself by a list of configuration parameters assigned by type, and each of the configuration parameters is validated by that type.
67
75
The settings list is required:
68
76
69
-
* validSettings | Defines the list of valid settings for this plugin
77
+
* validSettings: Defines the list of valid settings for this plugin
70
78
71
79
The following lists are optional and likely to be used by many plugins:
72
80
73
-
* intSettings | Will validate the settings as integers
74
-
* floatSettings | Will validate the settings as floating point numbers
75
-
* boolSettings | Will validate the settings as booleans
76
-
* jsonSettings | Will validate the settings as a JSON string
77
-
* defaultableSettings | Settings which can be specified in the [global] stanza and will pass down to individual stanzas
78
-
* complexSettings | A dictionary of lists or function callbacks, containing a setting name with list of valid options or a callback function to validate the setting.
81
+
* intSettings: Will validate the settings as integers
82
+
* floatSettings: Will validate the settings as floating point numbers
83
+
* boolSettings: Will validate the settings as booleans
84
+
* jsonSettings: Will validate the settings as a JSON string
85
+
* defaultableSettings: Settings which can be specified in the [global] stanza and will pass down to individual stanzas
86
+
* complexSettings: A dictionary of lists or function callbacks, containing a setting name with list of valid options or a callback function to validate the setting.
79
87
80
88
## Methods required per plugin type
81
89
82
90
Each plugin type will define a different method required.
Rater | ``rate()`` | Integer count of events to generate | n/a
94
+
Rater | ``rate()`` | Integer count of events to generate | N/A
87
95
Generator | ``gen(count, earliest, latest) `` | Success (0) | Events get put into an output queue by calling the Sample's ``send()`` or ``bulksend()`` methods in the output object.
88
96
Output | ``flush(q)`` | Success (0) | Gets a deque list q to operate upon and output as configured.
89
97
@@ -92,48 +100,46 @@ Output | ``flush(q)`` | Success (0) | Gets a deque list q to operate upon and ou
92
100
We reviewed a simple Output Plugin earlier, let's look at a simple Generator Plugin:
93
101
94
102
```python
95
-
from__future__import division
103
+
import datetime
104
+
from datetime import timedelta
105
+
96
106
from generatorplugin import GeneratorPlugin
97
-
import os
98
-
import logging
99
-
import datetime, time
100
-
import itertools
101
-
from collections import deque
107
+
from logging_config import logger
108
+
102
109
103
110
classWindbagGenerator(GeneratorPlugin):
104
111
def__init__(self, sample):
105
112
GeneratorPlugin.__init__(self, sample)
106
113
107
-
# Logger already setup by config, just get an instance
108
-
logger = logging.getLogger('eventgen')
109
-
globals()['logger'] = logger
110
-
111
-
from eventgenconfig import Config
112
-
globals()['c'] = Config()
113
-
114
-
defgen(self, count, earliest, latest):
115
-
l = [ {'_raw': '2014-01-05 23:07:08 WINDBAG Event 1 of 100000'} for i inxrange(count) ]
current_time_object = earliest + datetime.timedelta(0, time_interval * (i +1))
121
+
msg ='{0} -0700 WINDBAG Event {1} of {2}'.format(current_time_object, (i +1), count)
122
+
self._out.send(msg)
118
123
return0
119
124
125
+
120
126
defload():
121
127
return WindbagGenerator
128
+
122
129
```
123
130
124
-
For this generator plugin, notice we inherit from GeneratorPlugin instead of OutputPlugin. This plugin is also quite simple.
125
-
In its ``__init__()`` method, it calls the superclass ``__init__()`` and it sets up two global variables, c, which holds the config
126
-
(and is a Singleton pattern which can be instantiated many times) and a copy of the logger which we'll use for logging in most plugins.
131
+
For this generator plugin, notice we inherit from `GeneratorPlugin` instead of `OutputPlugin`. This plugin is also quite simple.
127
132
128
-
Secondly, it defines a gen() method, which generates ``count`` events between ``earliest`` and ``latest`` time. In this case, we ignore the timestamp and return just event text.
129
-
Then we call bulksend. This plugin has several performance optimizations: using a list constructor instead of a loop and using bulksend instead of send.
133
+
Secondly, it defines a `gen()` method, which generates ``count`` events between ``earliest`` and ``latest`` time. In this case, we ignore the timestamp and return just event text.
134
+
Then we call `bulksend`. This plugin has several performance optimizations: using a list constructor instead of a loop and using bulksend instead of send.
130
135
Let's see how this could be implemented in a slightly less performant but easier to understand way:
131
136
132
137
```python
133
-
defgen(self, count, earliest, latest):
134
-
for x inxrange(count):
135
-
self._sample.send({ '_raw': '2014-01-05 23:07:08 WINDBAG Event 1 of 100000' })
current_time_object = earliest + datetime.timedelta(0, time_interval * (i +1))
141
+
msg ='{0} -0700 WINDBAG Event {1} of {2}'.format(current_time_object, (i +1), count)
142
+
self._out.send(msg)
137
143
return0
138
144
```
139
145
@@ -142,4 +148,4 @@ Here, we use ``send()`` instead of ``bulksend()`` and a loop to make it easier t
142
148
# Shipping a Plugin
143
149
144
150
When you've developed a plugin that you want to use in your app, shipping it with your app is easy.
145
-
Place any Eventgen plugin in your Splunk app's ``bin/`` directory and we'll search for and find any plugins referenced by a ``outputMode``, ``generator`` or ``rater`` config statement.
151
+
Place any Eventgen plugin in your Splunk app's ``bin/`` directory and we'll search for and find any plugins referenced by a ``outputMode``, ``generator`` or ``rater`` config statement.
0 commit comments