Skip to content

Commit 51eb7da

Browse files
Cyan4973Cyan4973
authored
Merge pull request #4184 from facebook/ZSTD_getErrorCode
elevated ZSTD_getErrorCode() to stable status
2 parents 15c2916 + 2e02cd3 commit 51eb7da

File tree

5 files changed

+141
-101
lines changed

5 files changed

+141
-101
lines changed

‎build/single_file_libs/build_library_test.sh

+1
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ echo "Single file library creation script: PASSED"
7070

7171
# Copy the header to here (for the tests)
7272
cp "$ZSTD_SRC_ROOT/zstd.h" examples/zstd.h
73+
cp "$ZSTD_SRC_ROOT/zstd_errors.h" examples/zstd_errors.h
7374

7475
# Compile the generated output
7576
cc -Wall -Wextra -Werror -Wshadow -pthread -I. -Os -g0 -o $OUT_FILE zstd.c examples/roundtrip.c

‎contrib/gen_html/gen_html.cpp

+1
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,7 @@ int main(int argc, char *argv[]) {
211211

212212
ostream << "<html>\n<head>\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=ISO-8859-1\">\n<title>" << version << "</title>\n</head>\n<body>" << endl;
213213
ostream << "<h1>" << version << "</h1>\n";
214+
ostream << "Note: the content of this file has been automatically generated by parsing \"zstd.h\" \n";
214215

215216
ostream << "<hr>\n<a name=\"Contents\"></a><h2>Contents</h2>\n<ol>\n";
216217
for (size_t i=0; i<chapters.size(); i++)

‎doc/zstd_manual.html

+111-75
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,17 @@
11
<html>
22
<head>
33
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
4-
<title>zstd 1.5.6 Manual</title>
4+
<title>zstd 1.5.7 Manual</title>
55
</head>
66
<body>
7-
<h1>zstd 1.5.6 Manual</h1>
7+
<h1>zstd 1.5.7 Manual</h1>
8+
Note: the content of this file has been automatically generated by parsing "zstd.h"
89
<hr>
910
<a name="Contents"></a><h2>Contents</h2>
1011
<ol>
1112
<li><a href="#Chapter1">Introduction</a></li>
1213
<li><a href="#Chapter2">Version</a></li>
13-
<li><a href="#Chapter3">Simple API</a></li>
14+
<li><a href="#Chapter3">Simple Core API</a></li>
1415
<li><a href="#Chapter4">Explicit context</a></li>
1516
<li><a href="#Chapter5">Advanced compression API (Requires v1.4.0+)</a></li>
1617
<li><a href="#Chapter6">Advanced decompression API (Requires v1.4.0+)</a></li>
@@ -74,7 +75,7 @@ <h1>zstd 1.5.6 Manual</h1>
7475
</b><p> Return runtime library version, like "1.4.5". Requires v1.3.0+.
7576
</p></pre><BR>
7677

77-
<a name="Chapter3"></a><h2>Simple API</h2><pre></pre>
78+
<a name="Chapter3"></a><h2>Simple Core API</h2><pre></pre>
7879

7980
<pre><b>size_t ZSTD_compress( void* dst, size_t dstCapacity,
8081
const void* src, size_t srcSize,
@@ -88,38 +89,42 @@ <h1>zstd 1.5.6 Manual</h1>
8889

8990
<pre><b>size_t ZSTD_decompress( void* dst, size_t dstCapacity,
9091
const void* src, size_t compressedSize);
91-
</b><p> `compressedSize` : must be the _exact_ size of some number of compressed and/or skippable frames.
92-
`dstCapacity` is an upper bound of originalSize to regenerate.
93-
If user cannot imply a maximum upper bound, it's better to use streaming mode to decompress data.
94-
@return : the number of bytes decompressed into `dst` (<= `dstCapacity`),
95-
or an errorCode if it fails (which can be tested using ZSTD_isError()).
92+
</b><p> `compressedSize` : must be the _exact_ size of some number of compressed and/or skippable frames.
93+
Multiple compressed frames can be decompressed at once with this method.
94+
The result will be the concatenation of all decompressed frames, back to back.
95+
`dstCapacity` is an upper bound of originalSize to regenerate.
96+
First frame's decompressed size can be extracted using ZSTD_getFrameContentSize().
97+
If maximum upper bound isn't known, prefer using streaming mode to decompress data.
98+
@return : the number of bytes decompressed into `dst` (<= `dstCapacity`),
99+
or an errorCode if it fails (which can be tested using ZSTD_isError()).
96100
</p></pre><BR>
97101

102+
<h3>Decompression helper functions</h3><pre></pre><b><pre></pre></b><BR>
98103
<pre><b>#define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1)
99104
#define ZSTD_CONTENTSIZE_ERROR (0ULL - 2)
100105
unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize);
101-
</b><p> `src` should point to the start of a ZSTD encoded frame.
102-
`srcSize` must be at least as large as the frame header.
103-
hint : any size >= `ZSTD_frameHeaderSize_max` is large enough.
104-
@return : - decompressed size of `src` frame content, if known
105-
- ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
106-
- ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small)
107-
note 1 : a 0 return value means the frame is valid but "empty".
108-
note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode.
109-
When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
110-
In which case, it's necessary to use streaming mode to decompress data.
111-
Optionally, application can rely on some implicit limit,
112-
as ZSTD_decompress() only needs an upper bound of decompressed size.
113-
(For example, data could be necessarily cut into blocks <= 16 KB).
114-
note 3 : decompressed size is always present when compression is completed using single-pass functions,
115-
such as ZSTD_compress(), ZSTD_compressCCtx() ZSTD_compress_usingDict() or ZSTD_compress_usingCDict().
116-
note 4 : decompressed size can be very large (64-bits value),
117-
potentially larger than what local system can handle as a single memory segment.
118-
In which case, it's necessary to use streaming mode to decompress data.
119-
note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified.
120-
Always ensure return value fits within application's authorized limits.
121-
Each application can set its own limits.
122-
note 6 : This function replaces ZSTD_getDecompressedSize()
106+
</b><p> `src` should point to the start of a ZSTD encoded frame.
107+
`srcSize` must be at least as large as the frame header.
108+
hint : any size >= `ZSTD_frameHeaderSize_max` is large enough.
109+
@return : - decompressed size of `src` frame content, if known
110+
- ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
111+
- ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small)
112+
note 1 : a 0 return value means the frame is valid but "empty".
113+
note 2 : decompressed size is an optional field, it may not be present (typically in streaming mode).
114+
When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
115+
In which case, it's necessary to use streaming mode to decompress data.
116+
Optionally, application can rely on some implicit limit,
117+
as ZSTD_decompress() only needs an upper bound of decompressed size.
118+
(For example, data could be necessarily cut into blocks <= 16 KB).
119+
note 3 : decompressed size is always present when compression is completed using single-pass functions,
120+
such as ZSTD_compress(), ZSTD_compressCCtx() ZSTD_compress_usingDict() or ZSTD_compress_usingCDict().
121+
note 4 : decompressed size can be very large (64-bits value),
122+
potentially larger than what local system can handle as a single memory segment.
123+
In which case, it's necessary to use streaming mode to decompress data.
124+
note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified.
125+
Always ensure return value fits within application's authorized limits.
126+
Each application can set its own limits.
127+
note 6 : This function replaces ZSTD_getDecompressedSize()
123128
</p></pre><BR>
124129

125130
<pre><b>ZSTD_DEPRECATED("Replaced by ZSTD_getFrameContentSize")
@@ -140,50 +145,54 @@ <h1>zstd 1.5.6 Manual</h1>
140145
or an error code if input is invalid
141146
</p></pre><BR>
142147

143-
<h3>Helper functions</h3><pre></pre><b><pre></b>/* ZSTD_compressBound() :<b>
144-
* maximum compressed size in worst case single-pass scenario.
145-
* When invoking `ZSTD_compress()` or any other one-pass compression function,
146-
* it's recommended to provide @dstCapacity >= ZSTD_compressBound(srcSize)
147-
* as it eliminates one potential failure scenario,
148-
* aka not enough room in dst buffer to write the compressed frame.
149-
* Note : ZSTD_compressBound() itself can fail, if @srcSize > ZSTD_MAX_INPUT_SIZE .
150-
* In which case, ZSTD_compressBound() will return an error code
151-
* which can be tested using ZSTD_isError().
152-
*
153-
* ZSTD_COMPRESSBOUND() :
154-
* same as ZSTD_compressBound(), but as a macro.
155-
* It can be used to produce constants, which can be useful for static allocation,
156-
* for example to size a static array on stack.
157-
* Will produce constant value 0 if srcSize too large.
158-
*/
159-
#define ZSTD_MAX_INPUT_SIZE ((sizeof(size_t)==8) ? 0xFF00FF00FF00FF00ULL : 0xFF00FF00U)
148+
<h3>Compression helper functions</h3><pre></pre><b><pre></pre></b><BR>
149+
<pre><b>#define ZSTD_MAX_INPUT_SIZE ((sizeof(size_t)==8) ? 0xFF00FF00FF00FF00ULL : 0xFF00FF00U)
160150
#define ZSTD_COMPRESSBOUND(srcSize) (((size_t)(srcSize) >= ZSTD_MAX_INPUT_SIZE) ? 0 : (srcSize) + ((srcSize)>>8) + (((srcSize) < (128<<10)) ? (((128<<10) - (srcSize)) >> 11) </b>/* margin, from 64 to 0 */ : 0)) /* this formula ensures that bound(A) + bound(B) <= bound(A+B) as long as A and B >= 128 KB */<b>
161151
size_t ZSTD_compressBound(size_t srcSize); </b>/*!< maximum compressed size in worst case single-pass scenario */<b>
152+
</b><p> maximum compressed size in worst case single-pass scenario.
153+
When invoking `ZSTD_compress()`, or any other one-pass compression function,
154+
it's recommended to provide @dstCapacity >= ZSTD_compressBound(srcSize)
155+
as it eliminates one potential failure scenario,
156+
aka not enough room in dst buffer to write the compressed frame.
157+
Note : ZSTD_compressBound() itself can fail, if @srcSize > ZSTD_MAX_INPUT_SIZE .
158+
In which case, ZSTD_compressBound() will return an error code
159+
which can be tested using ZSTD_isError().
160+
161+
ZSTD_COMPRESSBOUND() :
162+
same as ZSTD_compressBound(), but as a macro.
163+
It can be used to produce constants, which can be useful for static allocation,
164+
for example to size a static array on stack.
165+
Will produce constant value 0 if srcSize is too large.
166+
167+
</p></pre><BR>
168+
169+
<h3>Error helper functions</h3><pre></pre><b><pre>#include "zstd_errors.h" </b>/* list of errors */<b>
162170
</b>/* ZSTD_isError() :<b>
163171
* Most ZSTD_* functions returning a size_t value can be tested for error,
164172
* using ZSTD_isError().
165173
* @return 1 if error, 0 otherwise
166174
*/
167-
unsigned ZSTD_isError(size_t code); </b>/*!< tells if a `size_t` function result is an error code */<b>
168-
const char* ZSTD_getErrorName(size_t code); </b>/*!< provides readable string from an error code */<b>
169-
int ZSTD_minCLevel(void); </b>/*!< minimum negative compression level allowed, requires v1.4.0+ */<b>
170-
int ZSTD_maxCLevel(void); </b>/*!< maximum compression level available */<b>
171-
int ZSTD_defaultCLevel(void); </b>/*!< default compression level, specified by ZSTD_CLEVEL_DEFAULT, requires v1.5.0+ */<b>
175+
unsigned ZSTD_isError(size_t result); </b>/*!< tells if a `size_t` function result is an error code */<b>
176+
ZSTD_ErrorCode ZSTD_getErrorCode(size_t functionResult); </b>/* convert a result into an error code, which can be compared to error enum list */<b>
177+
const char* ZSTD_getErrorName(size_t result); </b>/*!< provides readable string from a function result */<b>
178+
int ZSTD_minCLevel(void); </b>/*!< minimum negative compression level allowed, requires v1.4.0+ */<b>
179+
int ZSTD_maxCLevel(void); </b>/*!< maximum compression level available */<b>
180+
int ZSTD_defaultCLevel(void); </b>/*!< default compression level, specified by ZSTD_CLEVEL_DEFAULT, requires v1.5.0+ */<b>
172181
</pre></b><BR>
173182
<a name="Chapter4"></a><h2>Explicit context</h2><pre></pre>
174183

175184
<h3>Compression context</h3><pre> When compressing many times,
176-
it is recommended to allocate a context just once,
185+
it is recommended to allocate a compression context just once,
177186
and reuse it for each successive compression operation.
178-
This will make workload friendlier for system's memory.
187+
This will make the workload easier for system's memory.
179188
Note : re-using context is just a speed / resource optimization.
180189
It doesn't change the compression ratio, which remains identical.
181-
Note 2 : In multi-threaded environments,
182-
use one different context per thread for parallel execution.
190+
Note 2: For parallel execution in multi-threaded environments,
191+
use one different context per thread .
183192

184193
</pre><b><pre>typedef struct ZSTD_CCtx_s ZSTD_CCtx;
185194
ZSTD_CCtx* ZSTD_createCCtx(void);
186-
size_t ZSTD_freeCCtx(ZSTD_CCtx* cctx); </b>/* accept NULL pointer */<b>
195+
size_t ZSTD_freeCCtx(ZSTD_CCtx* cctx); </b>/* compatible with NULL pointer */<b>
187196
</pre></b><BR>
188197
<pre><b>size_t ZSTD_compressCCtx(ZSTD_CCtx* cctx,
189198
void* dst, size_t dstCapacity,
@@ -194,7 +203,7 @@ <h3>Compression context</h3><pre> When compressing many times,
194203
this function compresses at the requested compression level,
195204
__ignoring any other advanced parameter__ .
196205
If any advanced parameter was set using the advanced API,
197-
they will all be reset. Only `compressionLevel` remains.
206+
they will all be reset. Only @compressionLevel remains.
198207

199208
</p></pre><BR>
200209

@@ -394,7 +403,8 @@ <h3>Decompression context</h3><pre> When decompressing many times,
394403
* ZSTD_c_stableOutBuffer
395404
* ZSTD_c_blockDelimiters
396405
* ZSTD_c_validateSequences
397-
* ZSTD_c_useBlockSplitter
406+
* ZSTD_c_blockSplitterLevel
407+
* ZSTD_c_splitAfterSequences
398408
* ZSTD_c_useRowMatchFinder
399409
* ZSTD_c_prefetchCDictTables
400410
* ZSTD_c_enableSeqProducerFallback
@@ -421,7 +431,8 @@ <h3>Decompression context</h3><pre> When decompressing many times,
421431
ZSTD_c_experimentalParam16=1013,
422432
ZSTD_c_experimentalParam17=1014,
423433
ZSTD_c_experimentalParam18=1015,
424-
ZSTD_c_experimentalParam19=1016
434+
ZSTD_c_experimentalParam19=1016,
435+
ZSTD_c_experimentalParam20=1017
425436
} ZSTD_cParameter;
426437
</b></pre><BR>
427438
<pre><b>typedef struct {
@@ -718,7 +729,7 @@ <h3>Streaming compression functions</h3><pre></pre><b><pre>typedef enum {
718729
<a name="Chapter9"></a><h2>Streaming decompression - HowTo</h2><pre>
719730
A ZSTD_DStream object is required to track streaming operations.
720731
Use ZSTD_createDStream() and ZSTD_freeDStream() to create/release resources.
721-
ZSTD_DStream objects can be reused multiple times.
732+
ZSTD_DStream objects can be re-employed multiple times.
722733

723734
Use ZSTD_initDStream() to start a new decompression operation.
724735
@return : recommended first input size
@@ -728,16 +739,21 @@ <h3>Streaming compression functions</h3><pre></pre><b><pre>typedef enum {
728739
The function will update both `pos` fields.
729740
If `input.pos < input.size`, some input has not been consumed.
730741
It's up to the caller to present again remaining data.
742+
731743
The function tries to flush all data decoded immediately, respecting output buffer size.
732744
If `output.pos < output.size`, decoder has flushed everything it could.
733-
But if `output.pos == output.size`, there might be some data left within internal buffers.,
745+
746+
However, when `output.pos == output.size`, it's more difficult to know.
747+
If @return > 0, the frame is not complete, meaning
748+
either there is still some data left to flush within internal buffers,
749+
or there is more input to read to complete the frame (or both).
734750
In which case, call ZSTD_decompressStream() again to flush whatever remains in the buffer.
735751
Note : with no additional input provided, amount of data flushed is necessarily <= ZSTD_BLOCKSIZE_MAX.
736752
@return : 0 when a frame is completely decoded and fully flushed,
737753
or an error code, which can be tested using ZSTD_isError(),
738754
or any other value > 0, which means there is still some decoding or flushing to do to complete current frame :
739755
the return value is a suggested next input size (just a hint for better latency)
740-
that will never request more than the remaining frame size.
756+
that will never request more than the remaining content of the compressed frame.
741757

742758
<BR></pre>
743759

@@ -763,9 +779,10 @@ <h3>Streaming decompression functions</h3><pre></pre><b><pre></pre></b><BR>
763779
Function will update both input and output `pos` fields exposing current state via these fields:
764780
- `input.pos < input.size`, some input remaining and caller should provide remaining input
765781
on the next call.
766-
- `output.pos < output.size`, decoder finished and flushed all remaining buffers.
767-
- `output.pos == output.size`, potentially uncflushed data present in the internal buffers,
768-
call ZSTD_decompressStream() again to flush remaining data to output.
782+
- `output.pos < output.size`, decoder flushed internal output buffer.
783+
- `output.pos == output.size`, unflushed data potentially present in the internal buffers,
784+
check ZSTD_decompressStream() @return value,
785+
if > 0, invoke it again to flush remaining data to output.
769786
Note : with no additional input, amount of data flushed <= ZSTD_BLOCKSIZE_MAX.
770787

771788
@return : 0 when a frame is completely decoded and fully flushed,
@@ -1311,19 +1328,37 @@ <h3>Streaming decompression functions</h3><pre></pre><b><pre></pre></b><BR>
13111328

13121329
</p></pre><BR>
13131330

1314-
<pre><b></b><p> Generate sequences using ZSTD_compress2(), given a source buffer.
1331+
<pre><b>ZSTD_DEPRECATED("For debugging only, will be replaced by ZSTD_extractSequences()")
1332+
ZSTDLIB_STATIC_API size_t
1333+
ZSTD_generateSequences(ZSTD_CCtx* zc,
1334+
ZSTD_Sequence* outSeqs, size_t outSeqsSize,
1335+
const void* src, size_t srcSize);
1336+
</b><p> WARNING: This function is meant for debugging and informational purposes ONLY!
1337+
Its implementation is flawed, and it will be deleted in a future version.
1338+
It is not guaranteed to succeed, as there are several cases where it will give
1339+
up and fail. You should NOT use this function in production code.
1340+
1341+
This function is deprecated, and will be removed in a future version.
1342+
1343+
Generate sequences using ZSTD_compress2(), given a source buffer.
1344+
1345+
@param zc The compression context to be used for ZSTD_compress2(). Set any
1346+
compression parameters you need on this context.
1347+
@param outSeqs The output sequences buffer of size @p outSeqsSize
1348+
@param outSeqsSize The size of the output sequences buffer.
1349+
ZSTD_sequenceBound(srcSize) is an upper bound on the number
1350+
of sequences that can be generated.
1351+
@param src The source buffer to generate sequences from of size @p srcSize.
1352+
@param srcSize The size of the source buffer.
13151353

13161354
Each block will end with a dummy sequence
13171355
with offset == 0, matchLength == 0, and litLength == length of last literals.
13181356
litLength may be == 0, and if so, then the sequence of (of: 0 ml: 0 ll: 0)
13191357
simply acts as a block delimiter.
13201358

1321-
@zc can be used to insert custom compression params.
1322-
This function invokes ZSTD_compress2().
1323-
1324-
The output of this function can be fed into ZSTD_compressSequences() with CCtx
1325-
setting of ZSTD_c_blockDelimiters as ZSTD_sf_explicitBlockDelimiters
1326-
@return : number of sequences generated
1359+
@returns The number of sequences generated, necessarily less than
1360+
ZSTD_sequenceBound(srcSize), or an error code that can be checked
1361+
with ZSTD_isError().
13271362

13281363
</p></pre><BR>
13291364

@@ -1512,13 +1547,14 @@ <h3>Streaming decompression functions</h3><pre></pre><b><pre></pre></b><BR>
15121547
#ifdef __GNUC__
15131548
__attribute__((__unused__))
15141549
#endif
1515-
ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL }; </b>/**< this constant defers to stdlib's functions */<b>
15161550
</b><p> These prototypes make it possible to pass your own allocation/free functions.
15171551
ZSTD_customMem is provided at creation time, using ZSTD_create*_advanced() variants listed below.
15181552
All allocation/free operations will be completed using these custom variants instead of regular <stdlib.h> ones.
15191553

15201554
</p></pre><BR>
15211555

1556+
<pre><b>ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL }; </b>/**< this constant defers to stdlib's functions */<b>
1557+
</b></pre><BR>
15221558
<pre><b>typedef struct POOL_ctx_s ZSTD_threadPool;
15231559
ZSTDLIB_STATIC_API ZSTD_threadPool* ZSTD_createThreadPool(size_t numThreads);
15241560
ZSTDLIB_STATIC_API void ZSTD_freeThreadPool (ZSTD_threadPool* pool); </b>/* accept NULL pointer */<b>

0 commit comments

Comments
 (0)