Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add example. Encoder closer to boundaries in best.
  • Loading branch information
klauspost committed Nov 15, 2022
commit 9bd70cdb05f2befbf001075bfb051f221e9ef811
15 changes: 15 additions & 0 deletions s2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -973,6 +973,21 @@ Valid blocks encoded *without* a dictionary can be decoded with any dictionary.
There are no checks whether the supplied dictionary is the correct for a block.
Because of this there is no overhead by using a dictionary.

## Example

This is the dictionary content. Elements are separated by `[]`.

Dictionary: `[0x0a][Yesterday 25 bananas were added to Benjamins brown bag]`.

Initial repeat offset is set at 10, which is the letter `2`.

Encoded `[LIT "10"][REPEAT len=10][LIT "hich"][MATCH off=50 len=6][MATCH off=31 len=6][MATCH off=61 len=10]`

Decoded: `[10][ bananas w][hich][ were ][brown ][were added]`

Output: `10 bananas which were brown were added`


## Streams

For streams each block can use the dictionary.
Expand Down
20 changes: 18 additions & 2 deletions s2/encode_best.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,15 @@ func encodeBlockBest(dst, src []byte, dict *Dict) (d int) {
}
m := match{offset: offset, s: s, length: 4 + offset, rep: rep}
s += 4
for s <= sLimit {
for s < len(src) {
if len(src)-s < 8 {
if src[s] == src[m.length] {
m.length++
s++
continue
}
break
}
if diff := load64(src, s) ^ load64(src, m.length); diff != 0 {
m.length += bits.TrailingZeros64(diff) >> 3
break
Expand Down Expand Up @@ -157,7 +165,15 @@ func encodeBlockBest(dst, src []byte, dict *Dict) (d int) {
}
m := match{offset: offset, s: s, length: 4 + candidate, rep: rep, dict: true}
s += 4
for s <= sLimit && m.length < len(dict.dict)-8 {
for s <= sLimit && m.length < len(dict.dict) {
if len(src)-s < 8 || len(dict.dict)-m.length < 8 {
if src[s] == dict.dict[m.length] {
m.length++
s++
continue
}
break
}
if diff := load64(src, s) ^ load64(dict.dict, m.length); diff != 0 {
m.length += bits.TrailingZeros64(diff) >> 3
break
Expand Down