OSDN > ソフトウェアを探す > るつぼ - Ruby実験室 > SCM > SVN > コミット

るつぼ - Ruby実験室

R/O
SSH
HTTPS

rutsubo: コミット

コミットメタ情報

リビジョン	71 (tree)
日時	2015-04-19 23:05:37
作者	dearblue

ログメッセージ

extlz4:

* セーフレベルを確認して例外を発生させる方針をやめ、汚染の伝搬のみとするように変更
* Raw API を Block API に名称変更
* LZ4::BlockEncoder#predict メソッドを追加
* LZ4::BlockEncoder#inspect メソッドを追加
* LZ4::BlockDecoder.linksize メソッドを追加
* 互換性のため以前の rawXXX で始まるメソッドをそのまま利用できる事を目的とした lib/extlz4/compat.rb を追加
* ext/frameapi.c の処理でオブジェクトの汚染状態を伝搬させるように修正
* ext/frameapi.c から predict の痕跡を削除
* LZ4::Encoder#initialize の引数解析処理を修正

outport を省略可能となるように変更
キーワード引数 level を削除し、第二引数として与えるように変更
キーワード引数 blocksize を追加
キーワード引数 blocklink を追加
キーワード引数 streamsum を checksum に変更

* LZ4::Encoder#outport、LZ4::Encoder#outport= メソッドを追加
* LZ4::Encoder#prefs_XXX メソッドによる圧縮設定値を確認できる機能を追加
* LZ4::Encoder#inspect メソッドを追加
* LZ4::Decoder#inport メソッドを追加
* LZ4::Decoder#prefs_XXX メソッドによる圧縮設定値を確認できる機能を追加
* LZ4::Decoder#inspect メソッドを追加
* いくつかの試験項目の追加と修正
* その他の修正

変更サマリ

modified: trunk/extlz4/README.md (diff)
added: trunk/extlz4/test/sampledata.rb (diff)
modified: trunk/extlz4/test/test_blockapi.rb (diff)
modified: trunk/extlz4/test/test_frameapi.rb (diff)
modified: trunk/extlz4/ext/blockapi.c (diff)
modified: trunk/extlz4/ext/frameapi.c (diff)
modified: trunk/extlz4/ext/extconf.rb (diff)
modified: trunk/extlz4/ext/extlz4.c (diff)
modified: trunk/extlz4/ext/depend (diff)
modified: trunk/extlz4/ext/extlz4.h (diff)
modified: trunk/extlz4/Rakefile (diff)
modified: trunk/extlz4/lib/extlz4.rb (diff)
modified: trunk/extlz4/lib/extlz4/oldstream.rb (diff)
added: trunk/extlz4/lib/extlz4/compat.rb (diff)
modified: trunk/extlz4/bin/extlz4 (diff)
modified: trunk/extlz4/HISTORY.ja.md (diff)

差分

--- trunk/extlz4/README.md (revision 70)

+++ trunk/extlz4/README.md (revision 71)

		@@ -6,7 +6,9 @@
6	6
7	7	LZ4 データストリームを圧縮・伸張できます。lz4-cli で扱うことが出来ます。
8	8
9		- $ dmesg \| ruby -r extlz4 -e 'LZ4.encode_file($stdin.binmode, $stdout.binmode)' \| lz4c -d \| more
	9	+``` shell:shell
	10	+$ dmesg \| ruby -r extlz4 -e 'LZ4.encode_file($stdin.binmode, $stdout.binmode)' \| lz4c -d \| more
	11	+```
10	12
11	13	ほかの ruby 向けの lz4 バインディングライブラリとしては KOMIYA Atsushi さんによる [lz4-ruby (http://rubygems.org/gems/lz4-ruby)](http://rubygems.org/gems/lz4-ruby) があります。
12	14

		@@ -13,24 +15,25 @@
13	15
14	16	## SUMMARY (概要)
15	17
16		-- PACKAGE NAME (名称): extlz4
17		-- AUTHOR (制作者): dearblue <dearblue@users.sourceforge.jp>
18		-- HOW TO INSTALL (インストール手順): `gem install extlz4`
19		-- VERSION (バージョン情報): 0.2
20		-- RELEASE QUALITY (品質): alpha
21		-- LICENSING (ライセンス): 2-clause BSD License (二条項 BSD ライセンス)
22		-- DEPENDENCY GEMS (依存する gem パッケージ):
23		- - none (なし)
24		-- DEPENDENCY EXTERNAL C LIBRARIES (依存する外部 C ライブラリ):
25		- - none (なし)
26		-- BUNDLED EXTERNAL C LIBRARIES (同梱される外部 C ライブラリ):
27		- - lz4 (Yann Collet さんによる) <http://code.google.com/p/lz4/> (r127)
28		-- REPORT ISSUE TO (問題の報告先): <http://sourceforge.jp/projects/rutsubo/ticket/>
	18	+ * PACKAGE NAME (名称): extlz4
	19	+ * AUTHOR (制作者): dearblue <dearblue@users.sourceforge.jp>
	20	+ * REPORT ISSUE TO (問題の報告先): <http://sourceforge.jp/projects/rutsubo/ticket/>
	21	+ * HOW TO INSTALL (インストール手順): `gem install extlz4`
	22	+ * VERSION (バージョン情報): 0.2
	23	+ * RELEASE QUALITY (品質): alpha
	24	+ * LICENSING (ライセンス): 2-clause BSD License (二条項 BSD ライセンス)
	25	+ * DEPENDENCY GEMS (依存する gem パッケージ):
	26	+ * none (なし)
	27	+ * DEPENDENCY EXTERNAL C LIBRARIES (依存する外部 C ライブラリ):
	28	+ * none (なし)
	29	+ * BUNDLED EXTERNAL C LIBRARIES (同梱される外部 C ライブラリ):
	30	+ * lz4 (Yann Collet さんによる) <http://code.google.com/p/lz4/> (r127)
29	31
30	32
31	33	## ATTENTIONS (注意事項)
32	34
33		-- Many documents are written in japanese.
	35	+ * Many documents are written in japanese.
	36	+
34	37	(ドキュメントの多くは日本語で記述されています)
35	38
36	39

		@@ -59,31 +62,29 @@
59	62
60	63	## FEATURES (機能)
61	64
62		-- Generic LZ4 streaming data process
63		- - Decode LZ4 streaming data: ``LZ4.decode``
64		- - Encode LZ4 streaming data: ``LZ4.encode``
65		-- Generic LZ4 streaming data file process
66		- - Decode LZ4 streaming data file: ``LZ4.decode_file``
67		- - Encode LZ4 streaming data file: ``LZ4.encode_file``
68		-- Primitive LZ4 data process
69		- - Decode LZ4 data: ``LZ4.raw_decode``
70		- - Encode LZ4 data: ``LZ4.raw_encode`` (supporting high compression level)
71		- - Streaming Decode LZ4 data: ``LZ4.raw_stream_decode`` and ``LZ4::RawStreamDecoder#update``
72		- - Streaming Encode LZ4 data: ``LZ4.raw_stream_encode`` and ``LZ4::RawStreamEncoder#update`` (supporting high compression level)
	65	+ * Generic LZ4 frame data process
	66	+ * Decode LZ4 Frame data : LZ4.decode
	67	+ * Encode LZ4 Frame data : LZ4.encode
	68	+ * Generic LZ4 frame data file process
	69	+ * Decode LZ4 Frame data file : LZ4.decode\_file
	70	+ * Encode LZ4 Frame data file : LZ4.encode\_file
	71	+ * LZ4 block data process
	72	+ * Decode LZ4 block data : LZ4.block\_decode
	73	+ * Encode LZ4 block data : LZ4.block\_encode (supporting high compression level)
	74	+ * Streaming Decode LZ4 block data : LZ4.block\_stream\_decode and LZ4::BlockDecoder#update
	75	+ * Streaming Encode LZ4 block data : LZ4.block\_stream\_encode and LZ4::BlockEncoder#update (supporting high compression level)
73	76
74	77
75		-## ABOUT SECURITY (セキュリティについて)
	78	+## ABOUT TAINT STATE AND SECURITY (汚染状態とセキュリティについて)
76	79
77		-extlz4 はセキュリティレベルとオブジェクトの汚染状態を確認し、禁止される処理を決定します。
	80	+extlz4 はオブジェクト間での汚染状態を一方向伝播します。
78	81
79		-セーフレベルが3未満であれば、禁止される処理はありません。
	82	+オブジェクトの汚染伝播については『入力 -> 出力』となり、
	83	+ストリーム処理の場合は『入力 -> 圧縮器・伸張器 -> 出力』というようになります。
80	84
81		-セーフレベルが3以上の場合、入力と出力 (ストリーム処理の場合はストリーム処理器が含まれる) のすべてが汚染状態でなければ禁止されます。
	85	+セキュリティレベルによる処理の拒否は行いません。
82	86
83		-いずれのセーフレベルにおいても、オブジェクト間で汚染状態が一方向伝播されます。
84		-オブジェクトの汚染伝播については『入力 -> 出力』となり、ストリーム処理の場合は『入力 -> 圧縮器・伸張器 -> 出力』というようになります。
85	87
86		-
87	88	## EXAMPLES (用例)
88	89
89	90	First, load extlz4. (最初に extlz4 を読み込んでください)

		@@ -104,13 +105,13 @@
104	105	compressed_data_string = LZ4.encode(uncompressed_data_string)
105	106	```
106	107
107		-### High compression encoding (高効率圧縮処理)
	108	+### High compression encoding (高圧縮処理)
108	109
109	110	``` ruby:ruby
110	111	compressed_data_string = LZ4.encode(uncompressed_data_string, 9)
111	112	```
112	113
113		-### Stream decoding
	114	+### Frame decoding
114	115
115	116	``` ruby:ruby
116	117	File.open("sample.txt.lz4", "rb") do \|file\|

		@@ -122,7 +123,7 @@
122	123	end
123	124	```
124	125
125		-### Stream encoding by high compression
	126	+### Frame encoding by high compression
126	127
127	128	``` ruby:ruby
128	129	File.open("sample.txt.lz4", "wb") do \|file\|

		@@ -133,7 +134,7 @@
133	134	end
134	135	```
135	136
136		-### Stream encoding without block
	137	+### Frame encoding without block
137	138
138	139	``` ruby:ruby
139	140	file = File.open("sample.txt.lz4", "wb")

		@@ -142,39 +143,54 @@
142	143	lz4.close # VERY IMPORTANT!
143	144	```
144	145
145		-### Raw data processing (high compression encoding and decoding)
	146	+### Block data processing (fast compression encoding and decoding)
146	147
147		- src = "abcdefg" * 100
148		- lz4data = LZ4.raw_encode(16, src)
149		- data = LZ4.raw_decode(lz4data)
150		- p src == data # => true
	148	+``` ruby:ruby
	149	+src = "abcdefg" * 100
	150	+lz4data = LZ4.block_encode(src)
	151	+data = LZ4.block_decode(lz4data)
	152	+p src == data # => true
	153	+```
151	154
152		-### Raw stream data processing (high compression encoding and decoding)
	155	+### Block data processing (high compression encoding and decoding)
153	156
154		- blocksize = 4 * 1024 # 4 KiB (REQEUIRED PARAMETER)
155		- ishighcompress = true # use high compression method (OPTIONAL PARAMETER)
156		- encoder = LZ4.raw_stream_encode(blocksize, ishighcompress)
	157	+``` ruby:ruby
	158	+src = "abcdefg" * 100
	159	+level = 8
	160	+lz4data = LZ4.block_encode(level, src)
	161	+data = LZ4.block_decode(lz4data)
	162	+p src == data # => true
	163	+```
157	164
158		- src = "abcdefg" * 100
159		- lz4data1 = encoder.update(16, src)
	165	+### Block stream data processing (high compression encoding and decoding)
160	166
161		- decoder = LZ4.raw_stream_decode # not required blocksize
	167	+``` ruby:ruby
	168	+level = 8 # (OPTIONAL PARAMETER)
	169	+predict = "abcdefg" # with preset dictionary (OPTIONAL PARAMETER)
	170	+encoder = LZ4.block_stream_encode(level, predict)
162	171
163		- data = decoder.update(lz4data1)
164		- p src == data # => true
	172	+src = "abcdefg" * 100
	173	+lz4data1 = encoder.update(src)
165	174
166		- lz4data2 = encoder.update(src) # default high compression level
167		- p "lz4data1.bytesize" => lz4data1.bytesize,
168		- "lz4data2.bytesize" => lz4data2.bytesize
	175	+decoder = LZ4.block_stream_decode(predict)
169	176
170		- data = decoder.update(lz4data2)
171		- p src == data # => true
	177	+data = decoder.update(lz4data1)
	178	+p src == data # => true
172	179
	180	+src2 = "ABCDEFG" * 100
	181	+lz4data2 = encoder.update(src2)
	182	+p "lz4data1.bytesize" => lz4data1.bytesize,
	183	+ "lz4data2.bytesize" => lz4data2.bytesize
173	184
174		-## おまけ
	185	+data = decoder.update(lz4data2)
	186	+p src2 == data # => true
	187	+```
175	188
176		-コマンドラインプログラムとして `extlz4` が追加されます。
177	189
	190	+## BONUS (おまけ)
	191	+
	192	+コマンドラインプログラムとして ``extlz4`` が追加されます。
	193	+
178	194	これは lz4 と同程度の機能を持ちます (車輪の再発明とも言う)。
179	195
180	196	とはいえ、引数のとり方を変えてあり、gzip のような形で利用できます。

--- trunk/extlz4/test/sampledata.rb (nonexistent)

+++ trunk/extlz4/test/sampledata.rb (revision 71)

		@@ -0,0 +1,15 @@
	1	+SMALLSIZE = 400
	2	+BIGSIZE = 12000000
	3	+
	4	+SAMPLES = {
	5	+ "empty" => "",
	6	+ "\\0 (small size)" => "\0".b * SMALLSIZE,
	7	+ "\\0 (big size)" => "\0".b * BIGSIZE,
	8	+ "\\xaa (small size)" => "\xaa".b * SMALLSIZE,
	9	+ "\\xaa (big size)" => "\xaa".b * BIGSIZE,
	10	+ "random (small size)" => OpenSSL::Random.random_bytes(SMALLSIZE),
	11	+ "random (big size)" => OpenSSL::Random.random_bytes(BIGSIZE),
	12	+}
	13	+
	14	+SAMPLES["freebsd ports index"] = File.read("/usr/ports/INDEX-10", mode: "rb") rescue nil # if on FreeBSD
	15	+SAMPLES["freebsd kernel"] = File.read("/boot/kernel/kernel", mode: "rb") rescue nil # if on FreeBSD

--- trunk/extlz4/test/test_blockapi.rb (revision 70)

+++ trunk/extlz4/test/test_blockapi.rb (revision 71)

		@@ -1,36 +1,71 @@
1	1	#!ruby
2	2
3	3	require "test-unit"
	4	+require "openssl" # for OpenSSL::Random.random_bytes
4	5	require "extlz4"
5		-require "openssl" # for OpenSSL::Random.random_bytes
6	6
7		-SMALLSIZE = 400
8		-BIGSIZE = 12000000
	7	+require_relative "sampledata"
9	8
10		-SAMPLES = [
11		- "",
12		- "\0".b * SMALLSIZE,
13		- "\0".b * BIGSIZE,
14		- "\xaa".b * SMALLSIZE,
15		- "\xaa".b * BIGSIZE,
16		- OpenSSL::Random.random_bytes(SMALLSIZE),
17		- OpenSSL::Random.random_bytes(BIGSIZE),
18		-]
	9	+class TestBlockAPI < Test::Unit::TestCase
	10	+ SAMPLES.each_pair do \|name, data\|
	11	+ define_method("test_block_encode_decode_sample:#{name}", -> {
	12	+ assert(data, LZ4.block_decode(LZ4.block_encode(data)))
	13	+ })
	14	+ end
19	15
20		-SAMPLES << File.read("/usr/ports/INDEX-10", mode: "rb") rescue nil # if on FreeBSD
21		-SAMPLES << File.read("/boot/kernel/kernel", mode: "rb") rescue nil # if on FreeBSD
	16	+ def test_block_encode
	17	+ buf = ""
	18	+ assert_kind_of(String, LZ4.block_encode(SAMPLES["\\0 (small size)"]))
	19	+ assert_kind_of(String, LZ4.block_encode(SAMPLES["\\0 (small size)"], 1000))
	20	+ assert_same(buf, LZ4.block_encode(SAMPLES["\\0 (small size)"], buf))
	21	+ assert_same(buf, LZ4.block_encode(SAMPLES["\\0 (small size)"], 1000, buf))
	22	+ assert_same(buf, LZ4.block_encode(nil, SAMPLES["\\0 (small size)"], 1000, buf))
22	23
23		-$stderr.puts "%s:%d: prepaired sample data (%d samples)\n" % [__FILE__, __LINE__, SAMPLES.size]
	24	+ # high compression
	25	+ assert_kind_of(String, LZ4.block_encode(0, SAMPLES["\\0 (small size)"]))
	26	+ assert_kind_of(String, LZ4.block_encode(0, SAMPLES["\\0 (small size)"], 1000))
	27	+ assert_same(buf, LZ4.block_encode(0, SAMPLES["\\0 (small size)"], buf))
	28	+ assert_same(buf, LZ4.block_encode(0, SAMPLES["\\0 (small size)"], 1000, buf))
	29	+ end
24	30
25		-class TestRawAPI < Test::Unit::TestCase
26		- def test_encode_decode
27		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[0])) == SAMPLES[0] }
28		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[2])) == SAMPLES[2] }
29		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[3])) == SAMPLES[3] }
30		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[4])) == SAMPLES[4] }
31		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[5])) == SAMPLES[5] }
32		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[6])) == SAMPLES[6] }
33		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[7])) == SAMPLES[7] } if SAMPLES.size > 7
34		- assert { LZ4.raw_decode(LZ4.raw_encode(SAMPLES[8])) == SAMPLES[8] } if SAMPLES.size > 8
	31	+ def test_block_encode_invalid_args
	32	+ src = SAMPLES["\\0 (small size)"]
	33	+ buf = ""
	34	+ assert_raise(ArgumentError) { LZ4.block_encode } # no arguments
	35	+ assert_raise(ArgumentError) { LZ4.block_encode(-1, src) } # wrong level
	36	+ assert_raise(TypeError) { LZ4.block_encode(100) } # source is not string
	37	+ assert_raise(TypeError) { LZ4.block_encode(nil) } # source is not string
	38	+ assert_raise(TypeError) { LZ4.block_encode(:bad_input) } # source is not string
	39	+ assert_raise(TypeError) { LZ4.block_encode(/bad-input/) } # source is not string
	40	+ assert_raise(RuntimeError) { LZ4.block_encode(src, "bad-destbuf".freeze) } # can't modify frozen String
	41	+ assert_raise(LZ4::Error) { LZ4.block_encode(src, 1) } # maxdest is too small
	42	+ assert_raise(LZ4::Error) { LZ4.block_encode(src, 1, buf) } # maxdest is too small
	43	+ assert_raise(TypeError) { LZ4.block_encode(src, "bad-maxsize", "a") } # "bad-maxsize" is not integer
35	44	end
	45	+
	46	+ def test_block_decode
	47	+ src = LZ4.block_encode(SAMPLES["\\0 (small size)"])
	48	+ buf = ""
	49	+ assert_kind_of(String, LZ4.block_decode(src))
	50	+ assert_kind_of(String, LZ4.block_decode(src, 1000))
	51	+ assert_same(buf, LZ4.block_decode(src, buf))
	52	+ assert_same(buf, LZ4.block_decode(src, 1000, buf))
	53	+ end
	54	+
	55	+ def test_block_decode_invalid_args
	56	+ src = LZ4.block_encode(SAMPLES["\\0 (small size)"])
	57	+ buf = ""
	58	+ assert_raise(ArgumentError) { LZ4.block_decode } # no arguments
	59	+ assert_raise(TypeError) { LZ4.block_decode(-1, src) } # do not given level
	60	+ assert_raise(TypeError) { LZ4.block_decode(100) } # source is not string
	61	+ assert_raise(TypeError) { LZ4.block_decode(nil) } # source is not string
	62	+ assert_raise(TypeError) { LZ4.block_decode(:bad_input) } # source is not string
	63	+ assert_raise(TypeError) { LZ4.block_decode(/bad-input/) } # source is not string
	64	+ assert_raise(RuntimeError) { LZ4.block_decode(src, "bad-destbuf".freeze) } # can't modify frozen String
	65	+ assert_raise(TypeError) { LZ4.block_decode(src, "bad-maxsize", "a") } # "bad-maxsize" is not integer
	66	+
	67	+ src2 = SAMPLES["\\xaa (small size)"]
	68	+ assert_raise(LZ4::Error) { LZ4.block_decode(src2) } # encounted invalid end of sequence
	69	+ assert_raise(LZ4::Error) { LZ4.block_decode(src2, 100000) } # max_dest_size is too small, or data is corrupted
	70	+ end
36	71	end

--- trunk/extlz4/test/test_frameapi.rb (revision 70)

+++ trunk/extlz4/test/test_frameapi.rb (revision 71)

		@@ -1,66 +1,60 @@
1	1	#!ruby
2	2
	3	+=begin
3	4	# 必要と思われる試験項目
4		-#
5		-# * frameapi の圧縮処理
6		-# * LZ4.encode
7		-# * LZ4.decode で伸長できるか
8		-# * lz4-cli で伸長できるか
9		-# * 汚染状態の伝搬
10		-# * security level
11		-# * LZ4.encode_file
12		-# * LZ4.decode_file で伸長できるか
13		-# * lz4-cli で伸長できるか
14		-# * frameapi の伸長処理
15		-# * LZ4.decode
16		-# * LZ4.decode_file
17		-# * LZ4.test_file
18		-#
19		-# * 試験で用いる試料
20		-# * /usr/ports/INDEX-10
21		-# * /boot/kernel/kernel
22		-# * 長さ 0 の空データ
23		-# * 0 で埋められた小さなデータ
24		-# * 0 で埋められたでかいデータ
25		-# * 0xaa で埋められた小さなデータ
26		-# * 0xaa で埋められたでかいデータ
27		-# * /dev/random (4000 bytes)
28		-# * /dev/random (12000000 bytes)
29		-# * 可能であれば数十 GB レベルのファイル
30	5
	6	+* LZ4.encode
	7	+ * LZ4.decode で伸長できるか
	8	+ * lz4-cli で伸長できるか
	9	+ * 汚染状態の伝搬
	10	+ * security level
	11	+* LZ4.encode_file
	12	+ * LZ4.decode_file で伸長できるか
	13	+ * lz4-cli で伸長できるか
	14	+* LZ4.decode
	15	+* LZ4.decode_file
	16	+* LZ4.test_file
	17	+
	18	+* 試験で用いる試料
	19	+ * /usr/ports/INDEX-10
	20	+ * /boot/kernel/kernel
	21	+ * 長さ 0 の空データ
	22	+ * 0 で埋められた小さなデータ
	23	+ * 0 で埋められたでかいデータ
	24	+ * 0xaa で埋められた小さなデータ
	25	+ * 0xaa で埋められたでかいデータ
	26	+ * /dev/random (4000 bytes)
	27	+ * /dev/random (12000000 bytes)
	28	+ * 可能であれば数十 GB レベルのファイル
	29	+=end
	30	+
31	31	require "test-unit"
	32	+require "openssl" # for OpenSSL::Random.random_bytes
32	33	require "extlz4"
33		-require "openssl" # for OpenSSL::Random.random_bytes
34		-#require "zlib" # for Zlib.crc32
35	34
36		-SMALLSIZE = 400
37		-BIGSIZE = 12000000
	35	+require_relative "sampledata"
38	36
39		-SAMPLES = [
40		- "",
41		- "\0".b * SMALLSIZE,
42		- "\0".b * BIGSIZE,
43		- "\xaa".b * SMALLSIZE,
44		- "\xaa".b * BIGSIZE,
45		- OpenSSL::Random.random_bytes(SMALLSIZE),
46		- OpenSSL::Random.random_bytes(BIGSIZE),
47		-]
	37	+class TestFrameAPI < Test::Unit::TestCase
	38	+ SAMPLES.each_pair do \|name, data\|
	39	+ define_method("test_encode_decode_sample:#{name}", -> {
	40	+ assert(data, LZ4.decode(LZ4.encode(data)))
	41	+ })
	42	+ end
48	43
49		-SAMPLES << File.read("/usr/ports/INDEX-10", mode: "rb") rescue nil # if on FreeBSD
50		-SAMPLES << File.read("/boot/kernel/kernel", mode: "rb") rescue nil # if on FreeBSD
	44	+ def test_encode_args
	45	+ assert_kind_of(LZ4::Encoder, LZ4.encode)
	46	+ assert_kind_of(LZ4::Encoder, LZ4.encode(StringIO.new("")))
	47	+ assert_kind_of(String, LZ4.encode {})
	48	+ io = StringIO.new("")
	49	+ assert_same(io, LZ4.encode(io) {})
	50	+ assert_kind_of(LZ4::Encoder, LZ4.encode(io, 16))
	51	+ assert_kind_of(LZ4::Encoder, LZ4::Encoder.new)
	52	+ end
51	53
52		-$stderr.puts "%s:%d: prepaired sample data (%d samples)\n" % [__FILE__, __LINE__, SAMPLES.size]
53		-
54		-class TestFrameAPI < Test::Unit::TestCase
55		- def test_encode_decode
56		- assert { LZ4.decode(LZ4.encode(SAMPLES[0])) == SAMPLES[0] }
57		- assert { LZ4.decode(LZ4.encode(SAMPLES[1])) == SAMPLES[1] }
58		- assert { LZ4.decode(LZ4.encode(SAMPLES[2])) == SAMPLES[2] }
59		- assert { LZ4.decode(LZ4.encode(SAMPLES[3])) == SAMPLES[3] }
60		- assert { LZ4.decode(LZ4.encode(SAMPLES[4])) == SAMPLES[4] }
61		- assert { LZ4.decode(LZ4.encode(SAMPLES[5])) == SAMPLES[5] }
62		- assert { LZ4.decode(LZ4.encode(SAMPLES[6])) == SAMPLES[6] }
63		- assert { LZ4.decode(LZ4.encode(SAMPLES[7])) == SAMPLES[7] } if SAMPLES.size > 7
64		- assert { LZ4.decode(LZ4.encode(SAMPLES[8])) == SAMPLES[8] } if SAMPLES.size > 8
	54	+ def test_decode_args
	55	+ assert_raise(ArgumentError) { LZ4.decode }
	56	+ assert_raise(NoMethodError) { LZ4.decode(nil) } # undefined method `read' for nil:NilClass
	57	+ assert_raise(LZ4::Error) { LZ4.decode("") } # read error (or already EOF)
	58	+ assert_raise(ArgumentError) { LZ4.decode(nil, nil) } # wrong number of arguments (2 for 1)
65	59	end
66	60	end

--- trunk/extlz4/ext/blockapi.c (revision 70)

+++ trunk/extlz4/ext/blockapi.c (revision 71)

		@@ -2,7 +2,11 @@
2	2	#include <lz4.h>
3	3	#include <lz4hc.h>
4	4
5		-#if __GNUC__ \|\| __clang__
	5	+#define RDOCFAKE(code)
	6	+
	7	+RDOCFAKE(mLZ4 = rb_define_module("LZ4"));
	8	+
	9	+#if __GNUC__ \|\| __clang__ \|\| EXTLZ4_FORCE_EXPECT
6	10	#define AUX_LIKELY(x) __builtin_expect(!!(x), 1)
7	11	#define AUX_UNLIKELY(x) __builtin_expect(!!(x), 0)
8	12	#else

		@@ -10,50 +14,63 @@
10	14	#define AUX_UNLIKELY(x) (x)
11	15	#endif
12	16
13		-static inline const char *
14		-aux_lz4_expandsize(const char p, const char end, size_t *size)
	17	+static inline size_t
	18	+aux_lz4_expandsize(const char *p, const char end, size_t size)
15	19	{
16		- while (AUX_LIKELY(p < end)) {
17		- int s = (uint8_t)*p ++;
18		- *size += s;
19		- if (AUX_LIKELY(s != 255)) { return p; }
	20	+ while (AUX_LIKELY(*p < end)) {
	21	+ int s = (uint8_t)(p) ++;
	22	+ size += s;
	23	+ if (AUX_LIKELY(s != 255)) {
	24	+ return size;
	25	+ }
20	26	}
21	27
22	28	rb_raise(eError, "encounted invalid end of sequence");
23	29	}
24	30
25		-static inline const char *
26		-aux_lz4_scanseq(const char p, const char end, size_t *size)
	31	+static inline size_t
	32	+aux_lz4_scanseq(const char p, const char end, size_t *linksize)
27	33	{
	34	+ size_t size = 0;
28	35	while (AUX_LIKELY(p < end)) {
29	36	uint8_t token = (uint8_t)*p ++;
30	37	size_t s = token >> 4;
31	38	if (AUX_LIKELY(s == 15)) {
32		- p = aux_lz4_expandsize(p, end, &s);
	39	+ s = aux_lz4_expandsize(&p, end, s);
33	40	}
34		- *size += s;
	41	+ size += s;
35	42	p += s;
36	43
37		- s = token & 0x0f;
38		- if (AUX_UNLIKELY(s == 0 && p == end)) {
39		- return p;
40		- }
41		-
42	44	if (AUX_UNLIKELY(p + 2 >= end)) {
	45	+ if (p == end) {
	46	+#if 0
	47	+ s = token & 0x0f;
	48	+ if (s != 0) {
	49	+ // TODO: raise? or do nothing?
	50	+ }
	51	+#endif
	52	+ return size;
	53	+ }
43	54	break;
44	55	}
45	56	size_t offset = (uint8_t)*p ++;
46	57	offset \|= ((size_t)((uint8_t)*p ++)) << 8;
	58	+ if (linksize) {
	59	+ ssize_t n = offset - size;
	60	+ if (AUX_UNLIKELY(n > 0 && n > (ssize_t)*linksize)) {
	61	+ *linksize = n;
	62	+ }
	63	+ }
47	64	#if 0
48	65	if (AUX_UNLIKELY(offset == 0)) {
49	66	rb_raise(eError, "offset is zero");
50	67	}
51	68	#endif
	69	+ s = token & 0x0f;
52	70	if (AUX_LIKELY(s == 15)) {
53		- p = aux_lz4_expandsize(p, end, &s);
	71	+ s = aux_lz4_expandsize(&p, end, s);
54	72	}
55		- s += 4;
56		- *size += s;
	73	+ size += s + 4;
57	74	}
58	75
59	76	rb_raise(eError, "encounted invalid end of sequence");

		@@ -67,13 +84,31 @@
67	84	static size_t
68	85	aux_lz4_scansize(VALUE str)
69	86	{
70		- const char *p = RSTRING_PTR(str);
71		- const char *end = p + RSTRING_LEN(str);
	87	+ const char *p;
	88	+ size_t size;
	89	+ RSTRING_GETMEM(str, p, size);
72	90
73		- size_t total = 0;
74		- aux_lz4_scanseq(p, end, &total);
	91	+ return aux_lz4_scanseq(p, p + size, NULL);
	92	+}
75	93
76		- return total;
	94	+/*
	95	+ * offset トークンがバッファの負の数を表しているか確認する。
	96	+ *
	97	+ * 戻り値はその最大距離を返す (負の数として見るならば最小値だが、絶対値に変換する)。
	98	+ *
	99	+ * 名称の link は LZ4 frame からとった。
	100	+ */
	101	+static size_t
	102	+aux_lz4_linksize(VALUE str)
	103	+{
	104	+ const char *p;
	105	+ size_t size;
	106	+ RSTRING_GETMEM(str, p, size);
	107	+
	108	+ size_t linksize = 0;
	109	+ aux_lz4_scanseq(p, p + size, &linksize);
	110	+
	111	+ return linksize;
77	112	}
78	113
79	114	static inline VALUE

		@@ -83,25 +118,6 @@
83	118	return obj;
84	119	}
85	120
86		-/*
87		- * Check the object and security
88		- *
89		- * - $SAFE < 3: Pass always
90		- * - $SAFE >= 3: Pass if all arguments, otherwise prevention
91		- */
92		-static inline void
93		-check_security(VALUE processor, VALUE src, VALUE dest)
94		-{
95		- if (rb_safe_level() < 3 \|\|
96		- ((NIL_P(processor) \|\| OBJ_TAINTED(processor)) &&
97		- OBJ_TAINTED(src) && OBJ_TAINTED(dest))) {
98		-
99		- return;
100		- }
101		-
102		- rb_insecure_operation();
103		-}
104		-
105	121	static inline size_t
106	122	aux_lz4_compressbound(VALUE src)
107	123	{

		@@ -108,35 +124,31 @@
108	124	return LZ4_compressBound(RSTRING_LEN(src));
109	125	}
110	126
111		-/*
112		- * call-seq:
113		- * compressbound(src) -> size
114		- *
115		- * Calcuration maximum size of encoded data in worst case.
116		- */
117		-static VALUE
118		-rawenc_s_compressbound(VALUE mod, VALUE src)
	127	+enum {
	128	+ MAX_PREDICT_SIZE = 65536,
	129	+};
	130	+
	131	+static inline VALUE
	132	+make_predict(VALUE predict)
119	133	{
120		- return SIZET2NUM(aux_lz4_compressbound(src));
121		-}
	134	+ if (NIL_P(predict)) {
	135	+ return Qnil;
	136	+ }
122	137
123		-/*
124		- * call-seq:
125		- * scansize(lz4_rawencoded_data) -> integer
126		- *
127		- * Scan raw lz4 data, and get decoded byte size.
128		- *
129		- * このメソッドは、raw_decode メソッドに max_dest_size なしで利用する場合の検証目的で利用できるようにしてあります。
130		- *
131		- * その他の有用な使い方があるのかは不明です。
132		- */
133		-static VALUE
134		-rawdec_s_scansize(VALUE mod, VALUE str)
135		-{
136		- Check_Type(str, RUBY_T_STRING);
137		- return SIZET2NUM(aux_lz4_scansize(str));
	138	+ rb_check_type(predict, RUBY_T_STRING);
	139	+ size_t size = RSTRING_LEN(predict);
	140	+ if (size == 0) {
	141	+ return Qnil;
	142	+ }
	143	+ if (size > MAX_PREDICT_SIZE) {
	144	+ predict = rb_str_subseq(predict, size - MAX_PREDICT_SIZE, MAX_PREDICT_SIZE);
	145	+ } else {
	146	+ predict = rb_str_dup(predict);
	147	+ }
	148	+ return rb_str_freeze(predict);
138	149	}
139	150
	151	+
140	152	/*
141	153	* calculate destination size from source data
142	154	*/

		@@ -143,7 +155,7 @@
143	155	typedef size_t aux_calc_destsize_f(VALUE src);
144	156
145	157	static inline void
146		-rawprocess_args(int argc, VALUE argv[], VALUE src, VALUE dest, size_t maxsize, int level, aux_calc_destsize_f *calcsize)
	158	+blockprocess_args(int argc, VALUE argv[], VALUE src, VALUE dest, size_t maxsize, int level, aux_calc_destsize_f *calcsize)
147	159	{
148	160	const VALUE *argend = argv + argc;
149	161	VALUE tmp;

		@@ -194,149 +206,29 @@
194	206	rb_error_arity(argc, 1, (level ? 4 : 3));
195	207	}
196	208
197		-/***********/
198		-
199		-typedef int aux_lz4_encoder_f(const char src, char dest, int srcsize, int maxsize, int level);
200		-
201		-static int
202		-aux_LZ4_compress_limitedOutput(const char src, char dest, int srcsize, int maxsize, int level)
203		-{
204		- return LZ4_compress_limitedOutput(src, dest, srcsize, maxsize);
205		-}
206		-
207	209	/*
208		- * call-seq:
209		- * encode(src) -> compressed string data
210		- * encode(src, max_dest_size) -> compressed string data
211		- * encode(src, dest) -> dest with compressed string data
212		- * encode(src, max_dest_size, dest) -> dest with compressed string data
213		- * encode(level, src) -> compressed string data
214		- * encode(level, src, max_dest_size) -> compressed string data
215		- * encode(level, src, dest) -> dest with compressed string data
216		- * encode(level, src, max_dest_size, dest) -> dest with compressed string data
	210	+ * Document-class: LZ4::BlockEncoder
217	211	*
218		- * Encode to raw LZ4 data.
219		- *
220		- * level を指定した場合、より圧縮処理に時間を掛けて圧縮効率を高めることが出来ます。
221		- *
222		- * 実装の都合上、圧縮関数は LZ4_compress_limitedOutput / LZ4_compressHC2_limitedOutput が使われます。
223		- *
224		- * [RETURN]
225		- * 圧縮されたデータが文字列として返ります。dest を指定した場合は、圧縮データを格納した dest を返します。
226		- *
227		- * 圧縮データには自身の終わりやデータ長が含まれていないため、伸張する際には余計なデータが付随していると正常に伸張できません。
228		- *
229		- * [src]
230		- * 圧縮対象となる文字列オブジェクトを指定します。
231		- *
232		- * [max_dest_size]
233		- * 出力バッファの最大バイト数を指定します。圧縮時にこれよりも多くのバッファ長が必要になった場合は例外が発生します。
234		- *
235		- * 省略時は src 長から最悪値が計算されます。dest が最初に確保できれば圧縮処理中に例外が発生することがありません。
236		- *
237		- * [dest]
238		- * 出力先とする文字列オブジェクトを指定します。
239		- *
240		- * max_dest_size が同時に指定されない場合、出力バッファの最大バイト長は src 長から最悪値が求められて調整されます。
241		- *
242		- * [level]
243		- * 圧縮レベルとして 0 から 16 までの整数で指定すると、高効率圧縮処理が行われます。
244		- *
245		- * 0 を指定した場合、LZ4 の規定値による高効率圧縮処理が行われます。
246		- *
247		- * nil を与えるか省略した場合、通常の圧縮処理が行われます。
	212	+ * このクラスは LZ4 Block API を扱うためのものです。
248	213	*/
249		-static VALUE
250		-rawenc_s_encode(int argc, VALUE argv[], VALUE lz4)
251		-{
252		- VALUE src, dest;
253		- size_t maxsize;
254		- int level;
255		- rawprocess_args(argc, argv, &src, &dest, &maxsize, &level, aux_lz4_compressbound);
256		- check_security(Qnil, src, dest);
257	214
258		- aux_lz4_encoder_f *encoder;
259		- if (level < 0) {
260		- encoder = aux_LZ4_compress_limitedOutput;
261		- } else {
262		- encoder = LZ4_compressHC2_limitedOutput;
263		- }
	215	+typedef void blockencoder_reset_f(void *context, int level);
	216	+typedef void *blockencoder_create_f(int level);
	217	+typedef int blockencoder_free_f(void *context);
	218	+typedef int blockencoder_loaddict_f(void context, const char dict, int dictsize);
	219	+typedef int blockencoder_savedict_f(void context, char dict, int dictsize);
	220	+typedef int blockencoder_update_f(void context, const char src, char *dest, int srcsize, int destsize);
	221	+typedef int blockencoder_update_unlinked_f(void context, const char src, char *dest, int srcsize, int destsize);
264	222
265		- size_t srcsize = RSTRING_LEN(src);
266		- if (srcsize > LZ4_MAX_INPUT_SIZE) {
267		- rb_raise(eError,
268		- "source size is too big for lz4 encode (given %zu, but max %zu bytes)",
269		- srcsize, (size_t)LZ4_MAX_INPUT_SIZE);
270		- }
271		- aux_str_reserve(dest, maxsize);
272		- rb_str_set_len(dest, 0);
273		- rb_obj_infect(dest, src);
274		-
275		- int size = encoder(RSTRING_PTR(src), RSTRING_PTR(dest), srcsize, maxsize, level);
276		- if (size <= 0) {
277		- rb_raise(eError,
278		- "failed LZ4 compress - maxsize is too small, or out of memory");
279		- }
280		-
281		- rb_str_set_len(dest, size);
282		-
283		- return dest;
284		-}
285		-
286		-/*
287		- * call-seq:
288		- * decode(src) -> decoded string data
289		- * decode(src, max_dest_size) -> decoded string data
290		- * decode(src, dest) -> dest with decoded string data
291		- * decode(src, max_dest_size, dest) -> dest with decoded string data
292		- *
293		- * Decode raw LZ4 data.
294		- *
295		- * 出力先は、max_dest_size が与えられていない場合、必要に応じて自動的に拡張されます。
296		- * この場合、いったん圧縮された LZ4 データを走査するため、事前に僅かな CPU 時間を必要とします。
297		- */
298		-static VALUE
299		-rawdec_s_decode(int argc, VALUE argv[], VALUE lz4)
	223	+struct blockencoder_traits
300	224	{
301		- VALUE src, dest;
302		- size_t maxsize;
303		- rawprocess_args(argc, argv, &src, &dest, &maxsize, NULL, aux_lz4_scansize);
304		- check_security(Qnil, src, dest);
305		-
306		- aux_str_reserve(dest, maxsize);
307		- rb_str_set_len(dest, 0);
308		- rb_obj_infect(dest, src);
309		-
310		- int size = LZ4_decompress_safe(RSTRING_PTR(src), RSTRING_PTR(dest), RSTRING_LEN(src), maxsize);
311		- if (size < 0) {
312		- rb_raise(eError,
313		- "failed LZ4_decompress_safe - max_dest_size is too small, or data is corrupted");
314		- }
315		-
316		- rb_str_set_len(dest, size);
317		-
318		- return dest;
319		-}
320		-
321		-/***********/
322		-
323		-typedef void rawencoder_reset_f(void *context, int level);
324		-typedef void *rawencoder_create_f(int level);
325		-typedef int rawencoder_free_f(void *context);
326		-typedef int rawencoder_loaddict_f(void context, const char dict, int dictsize);
327		-typedef int rawencoder_savedict_f(void context, char dict, int dictsize);
328		-typedef int rawencoder_update_f(void context, const char src, char *dest, int srcsize, int destsize);
329		-typedef int rawencoder_update_unlinked_f(void context, const char src, char *dest, int srcsize, int destsize);
330		-
331		-struct rawencoder_traits
332		-{
333		- rawencoder_reset_f *reset;
334		- rawencoder_create_f *create;
335		- rawencoder_free_f *free;
336		- rawencoder_loaddict_f *loaddict;
337		- rawencoder_savedict_f *savedict;
338		- rawencoder_update_f *update;
339		- rawencoder_update_unlinked_f *update_unlinked;
	225	+ blockencoder_reset_f *reset;
	226	+ blockencoder_create_f *create;
	227	+ blockencoder_free_f *free;
	228	+ blockencoder_loaddict_f *loaddict;
	229	+ blockencoder_savedict_f *savedict;
	230	+ blockencoder_update_f *update;
	231	+ blockencoder_update_unlinked_f *update_unlinked;
340	232	};
341	233
342	234	static void

		@@ -346,60 +238,50 @@
346	238	LZ4_resetStream(context);
347	239	}
348	240
349		-static const struct rawencoder_traits rawencoder_traits_std = {
350		- .reset = (rawencoder_reset_f *)aux_LZ4_resetStream,
351		- .create = (rawencoder_create_f *)LZ4_createStream,
352		- .free = (rawencoder_free_f *)LZ4_freeStream,
353		- .loaddict = (rawencoder_loaddict_f *)LZ4_loadDict,
354		- .savedict = (rawencoder_savedict_f *)LZ4_saveDict,
355		- .update = (rawencoder_update_f *)LZ4_compress_limitedOutput_continue,
356		- .update_unlinked = (rawencoder_update_f *)LZ4_compress_limitedOutput_withState,
	241	+static const struct blockencoder_traits blockencoder_traits_std = {
	242	+ .reset = (blockencoder_reset_f *)aux_LZ4_resetStream,
	243	+ .create = (blockencoder_create_f *)LZ4_createStream,
	244	+ .free = (blockencoder_free_f *)LZ4_freeStream,
	245	+ .loaddict = (blockencoder_loaddict_f *)LZ4_loadDict,
	246	+ .savedict = (blockencoder_savedict_f *)LZ4_saveDict,
	247	+ .update = (blockencoder_update_f *)LZ4_compress_limitedOutput_continue,
	248	+ .update_unlinked = (blockencoder_update_f *)LZ4_compress_limitedOutput_withState,
357	249	};
358	250
359		-static const struct rawencoder_traits rawencoder_traits_hc = {
360		- .reset = (rawencoder_reset_f *)LZ4_resetStreamHC,
361		- .create = (rawencoder_create_f *)LZ4_createStreamHC,
362		- .free = (rawencoder_free_f *)LZ4_freeStreamHC,
363		- .loaddict = (rawencoder_loaddict_f *)LZ4_loadDictHC,
364		- .savedict = (rawencoder_savedict_f *)LZ4_saveDictHC,
365		- .update = (rawencoder_update_f *)LZ4_compressHC_limitedOutput_continue,
366		- .update_unlinked = (rawencoder_update_f *)LZ4_compressHC_limitedOutput_withStateHC,
	251	+static const struct blockencoder_traits blockencoder_traits_hc = {
	252	+ .reset = (blockencoder_reset_f *)LZ4_resetStreamHC,
	253	+ .create = (blockencoder_create_f *)LZ4_createStreamHC,
	254	+ .free = (blockencoder_free_f *)LZ4_freeStreamHC,
	255	+ .loaddict = (blockencoder_loaddict_f *)LZ4_loadDictHC,
	256	+ .savedict = (blockencoder_savedict_f *)LZ4_saveDictHC,
	257	+ .update = (blockencoder_update_f *)LZ4_compressHC_limitedOutput_continue,
	258	+ .update_unlinked = (blockencoder_update_f *)LZ4_compressHC_limitedOutput_withStateHC,
367	259	};
368	260
369		-struct rawencoder
	261	+struct blockencoder
370	262	{
371	263	void *context;
372		- const struct rawencoder_traits *traits;
	264	+ const struct blockencoder_traits *traits;
373	265	VALUE predict;
	266	+ int level;
374	267	int prefixsize;
375		- char prefix[1 + (1 << 16)]; /* 64 KiB; LZ4_loadDict, LZ4_saveDict */
376		-
377		-////////
378		-// VALUE predict; /* preset-dictionary (used when next reset) */
379		-// VALUE buffer; /* entity of input buffer */
380		-// char inoff; / current offset of buffer */
381		-// const char intail; / end offset of buffer */
382		-// void lz4; / lz4 stream context */
383		-// size_t blocksize; /* stream block size (maximum size) */
384		-// struct rawencoder_traits *traits;
385		-// VALUE ishc; /* false: not hc / true: hc */
386		-////////
	268	+ char prefix[1 << 16]; /* 64 KiB; LZ4_loadDict, LZ4_saveDict */
387	269	};
388	270
389	271	static void
390		-rawenc_mark(void *pp)
	272	+blkenc_mark(void *pp)
391	273	{
392	274	if (pp) {
393		- struct rawencoder *p = pp;
	275	+ struct blockencoder *p = pp;
394	276	rb_gc_mark(p->predict);
395	277	}
396	278	}
397	279
398	280	static void
399		-rawenc_free(void *pp)
	281	+blkenc_free(void *pp)
400	282	{
401	283	if (pp) {
402		- struct rawencoder *p = pp;
	284	+ struct blockencoder *p = pp;
403	285	if (p->context && p->traits) {
404	286	p->traits->free(p->context);
405	287	}

		@@ -408,36 +290,36 @@
408	290	}
409	291	}
410	292
411		-static const rb_data_type_t rawencoder_type = {
412		- .wrap_struct_name = "extlz4.LZ4.RawEncoder",
413		- .function.dmark = rawenc_mark,
414		- .function.dfree = rawenc_free,
415		- /* .function.dsize = rawenc_size, */
	293	+static const rb_data_type_t blockencoder_type = {
	294	+ .wrap_struct_name = "extlz4.LZ4.BlockEncoder",
	295	+ .function.dmark = blkenc_mark,
	296	+ .function.dfree = blkenc_free,
	297	+ /* .function.dsize = blkenc_size, */
416	298	};
417	299
418	300	static VALUE
419		-rawenc_alloc(VALUE klass)
	301	+blkenc_alloc(VALUE klass)
420	302	{
421		- struct rawencoder *p;
422		- VALUE v = TypedData_Make_Struct(klass, struct rawencoder, &rawencoder_type, p);
	303	+ struct blockencoder *p;
	304	+ VALUE v = TypedData_Make_Struct(klass, struct blockencoder, &blockencoder_type, p);
423	305	p->predict = Qnil;
424	306	return v;
425	307	}
426	308
427		-static inline struct rawencoder *
	309	+static inline struct blockencoder *
428	310	getencoderp(VALUE enc)
429	311	{
430		- return getrefp(enc, &rawencoder_type);
	312	+ return getrefp(enc, &blockencoder_type);
431	313	}
432	314
433		-static inline struct rawencoder *
	315	+static inline struct blockencoder *
434	316	getencoder(VALUE enc)
435	317	{
436		- return getref(enc, &rawencoder_type);
	318	+ return getref(enc, &blockencoder_type);
437	319	}
438	320
439	321	static inline void
440		-rawenc_setup(int argc, VALUE argv[], int level, struct rawencoder p)
	322	+blkenc_setup(int argc, VALUE argv[], struct blockencoder *p)
441	323	{
442	324	if (p->context) {
443	325	void *cx = p->context;

		@@ -449,25 +331,23 @@
449	331	rb_scan_args(argc, argv, "02", &level1, &p->predict);
450	332
451	333	if (NIL_P(level1)) {
452		- *level = -1;
453		- p->traits = &rawencoder_traits_std;
	334	+ p->level = -1;
	335	+ p->traits = &blockencoder_traits_std;
454	336	} else {
455		- *level = NUM2UINT(level1);
456		- p->traits = &rawencoder_traits_hc;
	337	+ p->level = NUM2UINT(level1);
	338	+ p->traits = &blockencoder_traits_hc;
457	339	}
458	340
459	341	if (argc < 2) {
460	342	p->predict = Qundef;
461	343	} else {
462		- if (!NIL_P(p->predict)) {
463		- rb_check_type(p->predict, RUBY_T_STRING);
464		- }
	344	+ p->predict = make_predict(p->predict);
465	345	}
466	346
467		- p->context = p->traits->create(*level);
	347	+ p->context = p->traits->create(p->level);
468	348	if (!p->context) {
469	349	rb_gc();
470		- p->context = p->traits->create(*level);
	350	+ p->context = p->traits->create(p->level);
471	351	if (!p->context) {
472	352	errno = ENOMEM;
473	353	rb_sys_fail("failed context allocation by LZ4_createStream()");

		@@ -479,6 +359,9 @@
479	359	* call-seq:
480	360	* initialize(level = nil, predict = nil)
481	361	*
	362	+ * [INFECTION]
	363	+ * +self+ < +predict+
	364	+ *
482	365	* [RETURN]
483	366	* self
484	367	*

		@@ -488,12 +371,12 @@
488	371	* When given +0+ .. +15+, encode high compression.
489	372	*
490	373	* [predict]
491		- * Pre load dictionary.
	374	+ * Preset dictionary.
492	375	*/
493	376	static VALUE
494		-rawenc_init(int argc, VALUE argv[], VALUE enc)
	377	+blkenc_init(int argc, VALUE argv[], VALUE enc)
495	378	{
496		- struct rawencoder *p = getencoder(enc);
	379	+ struct blockencoder *p = getencoder(enc);
497	380	if (p->context) {
498	381	rb_raise(eError,
499	382	"already initialized - #<%s:%p>",

		@@ -500,8 +383,7 @@
500	383	rb_obj_classname(enc), (void *)enc);
501	384	}
502	385
503		- int level;
504		- rawenc_setup(argc, argv, &level, p);
	386	+ blkenc_setup(argc, argv, p);
505	387	p->prefixsize = p->traits->savedict(p->context, p->prefix, sizeof(p->prefix));
506	388	if (p->predict == Qundef) {
507	389	p->predict = Qnil;

		@@ -513,16 +395,20 @@
513	395	}
514	396
515	397	/*
516		- * update(src [, max_dest_size] [, dest]) -> dest
	398	+ * call-seq:
	399	+ * update(src, dest = "") -> dest
	400	+ * update(src, max_dest_size, dest = "") -> dest
	401	+ *
	402	+ * [INFECTION]
	403	+ * +dest+ < +self+ < +src+
517	404	*/
518	405	static VALUE
519		-rawenc_update(int argc, VALUE argv[], VALUE enc)
	406	+blkenc_update(int argc, VALUE argv[], VALUE enc)
520	407	{
521		- struct rawencoder *p = getencoder(enc);
	408	+ struct blockencoder *p = getencoder(enc);
522	409	VALUE src, dest;
523	410	size_t maxsize;
524		- rawprocess_args(argc, argv, &src, &dest, &maxsize, NULL, aux_lz4_compressbound);
525		- check_security(enc, src, dest);
	411	+ blockprocess_args(argc, argv, &src, &dest, &maxsize, NULL, aux_lz4_compressbound);
526	412	rb_obj_infect(enc, src);
527	413	rb_obj_infect(dest, enc);
528	414	char *srcp;

		@@ -535,7 +421,6 @@
535	421	rb_str_capacity(dest));
536	422	}
537	423	p->prefixsize = p->traits->savedict(p->context, p->prefix, sizeof(p->prefix));
538		-//fprintf(stderr, "%s:%d:%s: rawencoder.prefixsize=%d\n", __FILE__, __LINE__, __func__, p->prefixsize);
539	424	rb_str_set_len(dest, s);
540	425	return dest;
541	426	}

		@@ -545,12 +430,15 @@
545	430	* reset(level = nil) -> self
546	431	* reset(level, predict) -> self
547	432	*
548		- * Reset raw stream encoder.
	433	+ * [INFECTION]
	434	+ * +self+ < +predict+
	435	+ *
	436	+ * Reset block stream encoder.
549	437	*/
550	438	static VALUE
551		-rawenc_reset(int argc, VALUE argv[], VALUE enc)
	439	+blkenc_reset(int argc, VALUE argv[], VALUE enc)
552	440	{
553		- struct rawencoder *p = getencoder(enc);
	441	+ struct blockencoder *p = getencoder(enc);
554	442	if (!p->context) {
555	443	rb_raise(eError,
556	444	"not initialized yet - #<%s:%p>",

		@@ -558,8 +446,7 @@
558	446	}
559	447
560	448	VALUE predict = p->predict;
561		- int level;
562		- rawenc_setup(argc, argv, &level, p);
	449	+ blkenc_setup(argc, argv, p);
563	450	if (p->predict == Qundef) {
564	451	p->predict = predict;
565	452	}

		@@ -572,9 +459,9 @@
572	459	}
573	460
574	461	static VALUE
575		-rawenc_release(VALUE enc)
	462	+blkenc_release(VALUE enc)
576	463	{
577		- struct rawencoder *p = getencoder(enc);
	464	+ struct blockencoder *p = getencoder(enc);
578	465	if (p->traits && p->context) {
579	466	p->traits->free(p->context);
580	467	}

		@@ -585,32 +472,173 @@
585	472	return Qnil;
586	473	}
587	474
	475	+static VALUE
	476	+blkenc_predict(VALUE enc)
	477	+{
	478	+ return getencoder(enc)->predict;
	479	+}
	480	+
	481	+static VALUE
	482	+blkenc_inspect(VALUE enc)
	483	+{
	484	+ struct blockencoder *p = getencoderp(enc);
	485	+ if (p && p->context) {
	486	+ if (p->level < 0) {
	487	+ return rb_sprintf("#<%s:%p (fast compression)%s>",
	488	+ rb_obj_classname(enc), (void *)enc,
	489	+ (NIL_P(p->predict)) ? "" : " (with predict)");
	490	+ } else {
	491	+ return rb_sprintf("#<%s:%p (high compression %d)%s>",
	492	+ rb_obj_classname(enc), (void *)enc, p->level,
	493	+ (NIL_P(p->predict)) ? "" : " (with predict)");
	494	+ }
	495	+ } else {
	496	+ return rb_sprintf("#<%s:%p NOT INITIALIZED>",
	497	+ rb_obj_classname(enc), (void *)enc);
	498	+ }
	499	+}
	500	+
588	501	/*
589		- * class LZ4::RawDecoder
	502	+ * call-seq:
	503	+ * compressbound(src) -> size
	504	+ *
	505	+ * Calcuration maximum size of encoded data in worst case.
590	506	*/
	507	+static VALUE
	508	+blkenc_s_compressbound(VALUE mod, VALUE src)
	509	+{
	510	+ return SIZET2NUM(aux_lz4_compressbound(src));
	511	+}
591	512
592		-struct rawdecoder
	513	+typedef int aux_lz4_encoder_f(const char src, char dest, int srcsize, int maxsize, int level);
	514	+
	515	+static int
	516	+aux_LZ4_compress_limitedOutput(const char src, char dest, int srcsize, int maxsize, int level)
593	517	{
	518	+ return LZ4_compress_limitedOutput(src, dest, srcsize, maxsize);
	519	+}
	520	+
	521	+/*
	522	+ * call-seq:
	523	+ * encode(src, dest = "") -> dest with compressed string data
	524	+ * encode(src, max_dest_size, dest = "") -> dest with compressed string data
	525	+ * encode(level, src, dest = "") -> dest with compressed string data
	526	+ * encode(level, src, max_dest_size, dest = "") -> dest with compressed string data
	527	+ *
	528	+ * Encode to block LZ4 data.
	529	+ *
	530	+ * level を指定した場合、より圧縮処理に時間を掛けて圧縮効率を高めることが出来ます。
	531	+ *
	532	+ * 実装の都合上、圧縮関数は LZ4_compress_limitedOutput / LZ4_compressHC2_limitedOutput が使われます。
	533	+ *
	534	+ * [INFECTION]
	535	+ * +dest+ < +src+
	536	+ *
	537	+ * [RETURN]
	538	+ * 圧縮されたデータが文字列として返ります。dest を指定した場合は、圧縮データを格納した dest を返します。
	539	+ *
	540	+ * 圧縮データには自身の終わりやデータ長が含まれていないため、伸張する際には余計なデータが付随していると正常に伸張できません。
	541	+ *
	542	+ * [src]
	543	+ * 圧縮対象となる文字列オブジェクトを指定します。
	544	+ *
	545	+ * [max_dest_size]
	546	+ * 出力バッファの最大バイト数を指定します。圧縮時にこれよりも多くのバッファ長が必要になった場合は例外が発生します。
	547	+ *
	548	+ * 省略時は src 長から最悪値が計算されます。dest が最初に確保できれば圧縮処理中に例外が発生することがありません。
	549	+ *
	550	+ * [dest]
	551	+ * 出力先とする文字列オブジェクトを指定します。
	552	+ *
	553	+ * max_dest_size が同時に指定されない場合、出力バッファの最大バイト長は src 長から最悪値が求められて調整されます。
	554	+ *
	555	+ * [level]
	556	+ * 圧縮レベルとして 0 から 16 までの整数で指定すると、高効率圧縮処理が行われます。
	557	+ *
	558	+ * 0 を指定した場合、LZ4 の規定値による高効率圧縮処理が行われます。
	559	+ *
	560	+ * nil を与えるか省略した場合、通常の圧縮処理が行われます。
	561	+ */
	562	+static VALUE
	563	+blkenc_s_encode(int argc, VALUE argv[], VALUE lz4)
	564	+{
	565	+ VALUE src, dest;
	566	+ size_t maxsize;
	567	+ int level;
	568	+ blockprocess_args(argc, argv, &src, &dest, &maxsize, &level, aux_lz4_compressbound);
	569	+
	570	+ aux_lz4_encoder_f *encoder;
	571	+ if (level < 0) {
	572	+ encoder = aux_LZ4_compress_limitedOutput;
	573	+ } else {
	574	+ encoder = LZ4_compressHC2_limitedOutput;
	575	+ }
	576	+
	577	+ size_t srcsize = RSTRING_LEN(src);
	578	+ if (srcsize > LZ4_MAX_INPUT_SIZE) {
	579	+ rb_raise(eError,
	580	+ "source size is too big for lz4 encode (given %zu, but max %zu bytes)",
	581	+ srcsize, (size_t)LZ4_MAX_INPUT_SIZE);
	582	+ }
	583	+ aux_str_reserve(dest, maxsize);
	584	+ rb_str_set_len(dest, 0);
	585	+ rb_obj_infect(dest, src);
	586	+
	587	+ int size = encoder(RSTRING_PTR(src), RSTRING_PTR(dest), srcsize, maxsize, level);
	588	+ if (size <= 0) {
	589	+ rb_raise(eError,
	590	+ "failed LZ4 compress - maxsize is too small, or out of memory");
	591	+ }
	592	+
	593	+ rb_str_set_len(dest, size);
	594	+
	595	+ return dest;
	596	+}
	597	+
	598	+static void
	599	+init_blockencoder(void)
	600	+{
	601	+ VALUE cBlockEncoder = rb_define_class_under(mLZ4, "BlockEncoder", rb_cObject);
	602	+ rb_define_alloc_func(cBlockEncoder, blkenc_alloc);
	603	+ rb_define_method(cBlockEncoder, "initialize", RUBY_METHOD_FUNC(blkenc_init), -1);
	604	+ rb_define_method(cBlockEncoder, "reset", RUBY_METHOD_FUNC(blkenc_reset), -1);
	605	+ rb_define_method(cBlockEncoder, "update", RUBY_METHOD_FUNC(blkenc_update), -1);
	606	+ rb_define_method(cBlockEncoder, "release", RUBY_METHOD_FUNC(blkenc_release), 0);
	607	+ rb_define_method(cBlockEncoder, "predict", RUBY_METHOD_FUNC(blkenc_predict), 0);
	608	+ rb_define_method(cBlockEncoder, "inspect", RUBY_METHOD_FUNC(blkenc_inspect), 0);
	609	+ rb_define_alias(cBlockEncoder, "encode", "update");
	610	+ rb_define_alias(cBlockEncoder, "compress", "update");
	611	+ rb_define_alias(cBlockEncoder, "free", "release");
	612	+
	613	+ rb_define_singleton_method(cBlockEncoder, "compressbound", blkenc_s_compressbound, 1);
	614	+ rb_define_singleton_method(cBlockEncoder, "encode", blkenc_s_encode, -1);
	615	+ rb_define_alias(rb_singleton_class(cBlockEncoder), "compress", "encode");
	616	+}
	617	+
	618	+/*
	619	+ * class LZ4::BlockDecoder
	620	+ */
	621	+
	622	+struct blockdecoder
	623	+{
594	624	void *context;
595	625	VALUE predict;
596		- //int prefixsize;
597		- //char prefix[1 + (1 << 16)]; /* 64 KiB; LZ4_loadDict, LZ4_saveDict */
598	626	};
599	627
600	628	static void
601		-rawdec_mark(void *pp)
	629	+blkdec_mark(void *pp)
602	630	{
603	631	if (pp) {
604		- struct rawdecoder *p = pp;
	632	+ struct blockdecoder *p = pp;
605	633	rb_gc_mark(p->predict);
606	634	}
607	635	}
608	636
609	637	static void
610		-rawdec_free(void *pp)
	638	+blkdec_free(void *pp)
611	639	{
612	640	if (pp) {
613		- struct rawdecoder *p = pp;
	641	+ struct blockdecoder *p = pp;
614	642	if (p->context) {
615	643	LZ4_freeStreamDecode(p->context);
616	644	}

		@@ -618,36 +646,36 @@
618	646	}
619	647	}
620	648
621		-static const rb_data_type_t rawdecoder_type = {
622		- .wrap_struct_name = "extlz4.LZ4.RawDecoder",
623		- .function.dmark = rawdec_mark,
624		- .function.dfree = rawdec_free,
625		- /* .function.dsize = rawdec_size, */
	649	+static const rb_data_type_t blockdecoder_type = {
	650	+ .wrap_struct_name = "extlz4.LZ4.BlockDecoder",
	651	+ .function.dmark = blkdec_mark,
	652	+ .function.dfree = blkdec_free,
	653	+ /* .function.dsize = blkdec_size, */
626	654	};
627	655
628	656	static VALUE
629		-rawdec_alloc(VALUE klass)
	657	+blkdec_alloc(VALUE klass)
630	658	{
631		- struct rawdecoder *p;
632		- VALUE v = TypedData_Make_Struct(klass, struct rawdecoder, &rawdecoder_type, p);
	659	+ struct blockdecoder *p;
	660	+ VALUE v = TypedData_Make_Struct(klass, struct blockdecoder, &blockdecoder_type, p);
633	661	p->predict = Qnil;
634	662	return v;
635	663	}
636	664
637		-static inline struct rawdecoder *
	665	+static inline struct blockdecoder *
638	666	getdecoderp(VALUE dec)
639	667	{
640		- return getrefp(dec, &rawdecoder_type);
	668	+ return getrefp(dec, &blockdecoder_type);
641	669	}
642	670
643		-static inline struct rawdecoder *
	671	+static inline struct blockdecoder *
644	672	getdecoder(VALUE dec)
645	673	{
646		- return getref(dec, &rawdecoder_type);
	674	+ return getref(dec, &blockdecoder_type);
647	675	}
648	676
649	677	static inline void
650		-rawdec_setup(int argc, VALUE argv[], VALUE predict, struct rawdecoder *p)
	678	+blkdec_setup(int argc, VALUE argv[], VALUE predict, struct blockdecoder *p)
651	679	{
652	680	VALUE predict1;
653	681	rb_scan_args(argc, argv, "01", &predict1);

		@@ -677,10 +705,13 @@
677	705	if (!NIL_P(predict)) {
678	706	if (LZ4_setStreamDecode(p->context, RSTRING_PTR(predict), RSTRING_LEN(predict)) == 0) {
679	707	rb_raise(eError,
680		- "failed set preset dictionary - LZ4_setStreamDecode()");
	708	+ "failed LZ4_setStreamDecode() with preset dictionary");
681	709	}
682	710	} else {
683		- LZ4_setStreamDecode(p->context, NULL, 0);
	711	+ if (LZ4_setStreamDecode(p->context, NULL, 0) == 0) {
	712	+ rb_raise(eError,
	713	+ "failed LZ4_setStreamDecode()");
	714	+ }
684	715	}
685	716	}
686	717

		@@ -688,13 +719,16 @@
688	719	* call-seq:
689	720	* initialize
690	721	* initialize(preset_dictionary)
	722	+ *
	723	+ * [INFECTION]
	724	+ * +self+ < +preset_dictionary+
691	725	*/
692	726	static VALUE
693		-rawdec_init(int argc, VALUE argv[], VALUE dec)
	727	+blkdec_init(int argc, VALUE argv[], VALUE dec)
694	728	{
695		- struct rawdecoder *p = getdecoder(dec);
	729	+ struct blockdecoder *p = getdecoder(dec);
696	730
697		- rawdec_setup(argc, argv, Qnil, p);
	731	+ blkdec_setup(argc, argv, Qnil, p);
698	732	rb_obj_infect(p->predict, dec);
699	733
700	734	return dec;

		@@ -704,13 +738,16 @@
704	738	* call-seq:
705	739	* reset
706	740	* reset(preset_dictionary)
	741	+ *
	742	+ * [INFECTION]
	743	+ * +self+ < +preset_dictionary+
707	744	*/
708	745	static VALUE
709		-rawdec_reset(int argc, VALUE argv[], VALUE dec)
	746	+blkdec_reset(int argc, VALUE argv[], VALUE dec)
710	747	{
711		- struct rawdecoder *p = getdecoder(dec);
	748	+ struct blockdecoder *p = getdecoder(dec);
712	749
713		- rawdec_setup(argc, argv, p->predict, p);
	750	+ blkdec_setup(argc, argv, p->predict, p);
714	751	rb_obj_infect(p->predict, dec);
715	752
716	753	return dec;

		@@ -718,31 +755,28 @@
718	755
719	756	/*
720	757	* call-seq:
721		- * update(src) -> decoded string data
722		- * update(src, max_dest_size) -> decoded string data
723		- * update(src, dest) -> dest for decoded string data
724		- * update(src, max_dest_size, dest) -> dest for decoded string data
	758	+ * update(src, dest = "") -> dest for decoded string data
	759	+ * update(src, max_dest_size, dest = "") -> dest for decoded string data
725	760	*
726		- * Decode raw lz4 data of stream block.
	761	+ * Decode block lz4 data of stream block.
727	762	*
728		- * Given arguments and return values are same as LZ4#raw_decode.
729		- * See LZ4#raw_decode for about its.
	763	+ * Given arguments and return values are same as LZ4#block_decode.
	764	+ * See LZ4#block_decode for about its.
730	765	*
731	766	* 出力先は、max_dest_size が与えられていない場合、必要に応じて自動的に拡張されます。
732	767	* この場合、いったん圧縮された LZ4 データを走査するため、事前に僅かな CPU 時間を必要とします。
733	768	*
734	769	* [INFECTION]
735		- * +src+ -> +self+ -> +dest+
	770	+ * +dest+ < +self+ < +src+
736	771	*/
737	772	static VALUE
738		-rawdec_update(int argc, VALUE argv[], VALUE dec)
	773	+blkdec_update(int argc, VALUE argv[], VALUE dec)
739	774	{
740		- struct rawdecoder *p = getdecoder(dec);
	775	+ struct blockdecoder *p = getdecoder(dec);
741	776	if (!p->context) { rb_raise(eError, "need reset (context not initialized)"); }
742	777	VALUE src, dest;
743	778	size_t maxsize;
744		- rawprocess_args(argc, argv, &src, &dest, &maxsize, NULL, aux_lz4_scansize);
745		- check_security(dec, src, dest);
	779	+ blockprocess_args(argc, argv, &src, &dest, &maxsize, NULL, aux_lz4_scansize);
746	780	rb_obj_infect(dec, src);
747	781	rb_obj_infect(dest, dec);
748	782	const char *srcp;

		@@ -764,9 +798,9 @@
764	798	* Release allocated internal heap memory.
765	799	*/
766	800	static VALUE
767		-rawdec_release(VALUE lz4)
	801	+blkdec_release(VALUE lz4)
768	802	{
769		- struct rawdecoder *p = getdecoderp(lz4);
	803	+ struct blockdecoder *p = getdecoderp(lz4);
770	804	if (!p) { return Qnil; }
771	805	if (p->context) {
772	806	LZ4_freeStreamDecode(p->context);

		@@ -778,36 +812,97 @@
778	812	}
779	813
780	814	/*
781		- * initializer rawapi.c
	815	+ * call-seq:
	816	+ * scansize(lz4_blockencoded_data) -> integer
	817	+ *
	818	+ * Scan block lz4 data, and get decoded byte size.
	819	+ *
	820	+ * このメソッドは、block_decode メソッドに max_dest_size なしで利用する場合の検証目的で利用できるようにしてあります。
	821	+ *
	822	+ * その他の有用な使い方があるのかは不明です。
782	823	*/
	824	+static VALUE
	825	+blkdec_s_scansize(VALUE mod, VALUE str)
	826	+{
	827	+ rb_check_type(str, RUBY_T_STRING);
	828	+ return SIZET2NUM(aux_lz4_scansize(str));
	829	+}
783	830
784		-void
785		-extlz4_init_rawapi(void)
	831	+/*
	832	+ * call-seq:
	833	+ * linksize(lz4_blockencoded_data) -> prefix size as integer
	834	+ *
	835	+ * Scan block lz4 data, and get prefix byte size.
	836	+ */
	837	+static VALUE
	838	+blkdec_s_linksize(VALUE mod, VALUE str)
786	839	{
787		- VALUE cRawEncoder = rb_define_class_under(mLZ4, "RawEncoder", rb_cObject);
788		- rb_define_singleton_method(cRawEncoder, "compressbound", rawenc_s_compressbound, 1);
789		- rb_define_singleton_method(cRawEncoder, "encode", rawenc_s_encode, -1);
790		- rb_define_alias(rb_singleton_class(cRawEncoder), "compress", "encode");
791		- rb_define_alloc_func(cRawEncoder, rawenc_alloc);
792		- rb_define_method(cRawEncoder, "initialize", RUBY_METHOD_FUNC(rawenc_init), -1);
793		- rb_define_method(cRawEncoder, "reset", RUBY_METHOD_FUNC(rawenc_reset), -1);
794		- rb_define_method(cRawEncoder, "update", RUBY_METHOD_FUNC(rawenc_update), -1);
795		- rb_define_method(cRawEncoder, "release", RUBY_METHOD_FUNC(rawenc_release), 0);
796		- rb_define_alias(cRawEncoder, "encode", "update");
797		- rb_define_alias(cRawEncoder, "compress", "update");
798		- rb_define_alias(cRawEncoder, "free", "release");
	840	+ rb_check_type(str, RUBY_T_STRING);
	841	+ return SIZET2NUM(aux_lz4_linksize(str));
	842	+}
799	843
800		- VALUE cRawDecoder = rb_define_class_under(mLZ4, "RawDecoder", rb_cObject);
801		- rb_define_singleton_method(cRawDecoder, "scansize", rawdec_s_scansize, 1);
802		- rb_define_singleton_method(cRawDecoder, "decode", rawdec_s_decode, -1);
803		- rb_define_alias(rb_singleton_class(cRawDecoder), "decompress", "decode");
804		- rb_define_alias(rb_singleton_class(cRawDecoder), "uncompress", "decode");
805		- rb_define_alloc_func(cRawDecoder, rawdec_alloc);
806		- rb_define_method(cRawDecoder, "initialize", RUBY_METHOD_FUNC(rawdec_init), -1);
807		- rb_define_method(cRawDecoder, "reset", RUBY_METHOD_FUNC(rawdec_reset), -1);
808		- rb_define_method(cRawDecoder, "update", RUBY_METHOD_FUNC(rawdec_update), -1);
809		- rb_define_method(cRawDecoder, "release", RUBY_METHOD_FUNC(rawdec_release), 0);
810		- rb_define_alias(cRawDecoder, "decode", "update");
811		- rb_define_alias(cRawDecoder, "decompress", "update");
812		- rb_define_alias(cRawDecoder, "uncompress", "update");
	844	+/*
	845	+ * call-seq:
	846	+ * decode(src, dest = "") -> dest with decoded string data
	847	+ * decode(src, max_dest_size, dest = "") -> dest with decoded string data
	848	+ *
	849	+ * Decode block LZ4 data.
	850	+ *
	851	+ * 出力先は、max_dest_size が与えられていない場合、必要に応じて自動的に拡張されます。
	852	+ * この場合、いったん圧縮された LZ4 データを走査するため、事前に僅かな CPU 時間を必要とします。
	853	+ *
	854	+ * [INFECTION]
	855	+ * +dest+ < +src+
	856	+ */
	857	+static VALUE
	858	+blkdec_s_decode(int argc, VALUE argv[], VALUE lz4)
	859	+{
	860	+ VALUE src, dest;
	861	+ size_t maxsize;
	862	+ blockprocess_args(argc, argv, &src, &dest, &maxsize, NULL, aux_lz4_scansize);
	863	+
	864	+ aux_str_reserve(dest, maxsize);
	865	+ rb_str_set_len(dest, 0);
	866	+ rb_obj_infect(dest, src);
	867	+
	868	+ int size = LZ4_decompress_safe(RSTRING_PTR(src), RSTRING_PTR(dest), RSTRING_LEN(src), maxsize);
	869	+ if (size < 0) {
	870	+ rb_raise(eError,
	871	+ "failed LZ4_decompress_safe - max_dest_size is too small, or data is corrupted");
	872	+ }
	873	+
	874	+ rb_str_set_len(dest, size);
	875	+
	876	+ return dest;
813	877	}
	878	+
	879	+static void
	880	+init_blockdecoder(void)
	881	+{
	882	+ VALUE cBlockDecoder = rb_define_class_under(mLZ4, "BlockDecoder", rb_cObject);
	883	+ rb_define_alloc_func(cBlockDecoder, blkdec_alloc);
	884	+ rb_define_method(cBlockDecoder, "initialize", RUBY_METHOD_FUNC(blkdec_init), -1);
	885	+ rb_define_method(cBlockDecoder, "reset", RUBY_METHOD_FUNC(blkdec_reset), -1);
	886	+ rb_define_method(cBlockDecoder, "update", RUBY_METHOD_FUNC(blkdec_update), -1);
	887	+ rb_define_method(cBlockDecoder, "release", RUBY_METHOD_FUNC(blkdec_release), 0);
	888	+ rb_define_alias(cBlockDecoder, "decode", "update");
	889	+ rb_define_alias(cBlockDecoder, "decompress", "update");
	890	+ rb_define_alias(cBlockDecoder, "uncompress", "update");
	891	+
	892	+ rb_define_singleton_method(cBlockDecoder, "scansize", blkdec_s_scansize, 1);
	893	+ rb_define_singleton_method(cBlockDecoder, "linksize", blkdec_s_linksize, 1);
	894	+ rb_define_singleton_method(cBlockDecoder, "decode", blkdec_s_decode, -1);
	895	+ rb_define_alias(rb_singleton_class(cBlockDecoder), "decompress", "decode");
	896	+ rb_define_alias(rb_singleton_class(cBlockDecoder), "uncompress", "decode");
	897	+}
	898	+
	899	+/*
	900	+ * initializer blockapi.c
	901	+ */
	902	+
	903	+void
	904	+extlz4_init_blockapi(void)
	905	+{
	906	+ init_blockencoder();
	907	+ init_blockdecoder();
	908	+}

--- trunk/extlz4/ext/frameapi.c (revision 70)

+++ trunk/extlz4/ext/frameapi.c (revision 71)

		@@ -1,7 +1,9 @@
1	1	#include "extlz4.h"
	2	+#include <stdarg.h>
2	3	#include <lz4frame.h>
3	4	#include <lz4frame_static.h>
4	5	#include "hashargs.h"
	6	+#include <ruby/thread.h>
5	7
6	8	static ID id_op_lshift;
7	9	static ID id_read;

		@@ -32,6 +34,69 @@
32	34	}
33	35	}
34	36
	37	+static inline void *
	38	+aux_thread_call_without_gvl(void (func)(void ), void (cancel)(void *), ...)
	39	+{
	40	+ va_list va1, va2;
	41	+ va_start(va1, cancel);
	42	+ va_start(va2, cancel);
	43	+ void *s = rb_thread_call_without_gvl(func, &va1, cancel, &va2);
	44	+ va_end(va1);
	45	+ va_end(va2);
	46	+ return s;
	47	+}
	48	+
	49	+static void *
	50	+aux_LZ4F_compressUpdate_nogvl(void *pp)
	51	+{
	52	+ va_list *p = pp;
	53	+ LZ4F_compressionContext_t encoder = va_arg(p, LZ4F_compressionContext_t *);
	54	+ char dest = va_arg(p, char *);
	55	+ size_t destsize = va_arg(*p, size_t);
	56	+ const char src = va_arg(p, const char *);
	57	+ size_t srcsize = va_arg(*p, size_t);
	58	+ LZ4F_compressOptions_t opts = va_arg(p, LZ4F_compressOptions_t *);
	59	+
	60	+ return (void *)LZ4F_compressUpdate(encoder, dest, destsize, src, srcsize, opts);
	61	+}
	62	+
	63	+static size_t
	64	+aux_LZ4F_compressUpdate(LZ4F_compressionContext_t *encoder,
	65	+ char dest, size_t destsize, const char src, size_t srcsize,
	66	+ LZ4F_compressOptions_t *opts)
	67	+{
	68	+ return (size_t)aux_thread_call_without_gvl(aux_LZ4F_compressUpdate_nogvl, NULL,
	69	+ encoder, dest, destsize, src, srcsize, opts);
	70	+}
	71	+
	72	+static int
	73	+aux_frame_level(const LZ4F_preferences_t *p)
	74	+{
	75	+ return p->compressionLevel;
	76	+}
	77	+
	78	+static int
	79	+aux_frame_blocksize(const LZ4F_frameInfo_t *info)
	80	+{
	81	+ int bsid = info->blockSizeID;
	82	+ if (bsid == 0) {
	83	+ bsid = max4MB;
	84	+ }
	85	+ return 1 << (bsid * 2 + 8);
	86	+}
	87	+
	88	+static int
	89	+aux_frame_blocklink(const LZ4F_frameInfo_t *info)
	90	+{
	91	+ return info->blockMode == blockLinked;
	92	+}
	93	+
	94	+static int
	95	+aux_frame_checksum(const LZ4F_frameInfo_t *info)
	96	+{
	97	+ return info->contentChecksumFlag == contentChecksumEnabled;
	98	+}
	99	+
35	100	/* class LZ4::Encoder */
36	101
37	102	struct encoder

		@@ -38,7 +103,6 @@
38	103	{
39	104	VALUE outport;
40	105	VALUE workbuf;
41		- VALUE predict;
42	106	LZ4F_preferences_t prefs;
43	107	LZ4F_compressionContext_t encoder;
44	108	};

		@@ -50,7 +114,6 @@
50	114	struct encoder *p = pp;
51	115	rb_gc_mark(p->outport);
52	116	rb_gc_mark(p->workbuf);
53		- rb_gc_mark(p->predict);
54	117	}
55	118	}
56	119

		@@ -79,41 +142,76 @@
79	142	VALUE obj = TypedData_Make_Struct(mod, struct encoder, &encoder_type, p);
80	143	p->outport = Qnil;
81	144	p->workbuf = Qnil;
82		- p->predict = Qnil;
83	145	return obj;
84	146	}
85	147
	148	+static inline int
	149	+fenc_init_args_blocksize(size_t size)
	150	+{
	151	+ if (size == 0) {
	152	+ return max4MB;
	153	+ } else if (size <= 64 * 1024) {
	154	+ return max64KB;
	155	+ } else if (size <= 256 * 1024) {
	156	+ return max256KB;
	157	+ } else if (size <= 1 * 1024 * 1024) {
	158	+ return max1MB;
	159	+ } else {
	160	+ return max4MB;
	161	+ }
	162	+}
	163	+
86	164	static inline void
87	165	fenc_init_args(int argc, VALUE argv[], VALUE outport, LZ4F_preferences_t prefs)
88	166	{
89	167	VALUE level, opts;
90		- rb_scan_args(argc, argv, "11:", outport, &level, &opts);
	168	+ rb_scan_args(argc, argv, "02:", outport, &level, &opts);
91	169
	170	+ memset(prefs, 0, sizeof(*prefs));
	171	+
	172	+ if (NIL_P(*outport)) {
	173	+ *outport = rb_str_buf_new(0);
	174	+ }
	175	+
	176	+ prefs->compressionLevel = NIL_P(level) ? 1 : NUM2INT(level);
	177	+
92	178	if (!NIL_P(opts)) {
93		- VALUE blocklink, streamsum;
	179	+ VALUE blocksize, blocklink, checksum;
94	180	RBX_SCANHASH(opts, Qnil,
95		- RBX_SCANHASH_ARGS("level", &level, INT2FIX(1)),
	181	+ RBX_SCANHASH_ARGS("blocksize", &blocksize, Qnil),
96	182	RBX_SCANHASH_ARGS("blocklink", &blocklink, Qfalse),
97		- RBX_SCANHASH_ARGS("streamsum", &streamsum, Qtrue));
98		- memset(prefs, 0, sizeof(*prefs));
99		- // prefs->autoFlush = ????;
100		- //prefs->frameInfo.blockSizeID = ; /* max64KB, max256KB, max1MB, max4MB ; 0 == default */
	183	+ RBX_SCANHASH_ARGS("checksum", &checksum, Qtrue));
	184	+ // prefs->autoFlush = TODO;
	185	+ prefs->frameInfo.blockSizeID = NIL_P(blocksize) ? max4MB : fenc_init_args_blocksize(NUM2INT(blocksize));
101	186	prefs->frameInfo.blockMode = RTEST(blocklink) ? blockLinked : blockIndependent;
102		- prefs->frameInfo.contentChecksumFlag = RTEST(streamsum) ? contentChecksumEnabled : noContentChecksum;
	187	+ prefs->frameInfo.contentChecksumFlag = RTEST(checksum) ? contentChecksumEnabled : noContentChecksum;
103	188	} else {
104		- memset(prefs, 0, sizeof(*prefs));
	189	+ prefs->frameInfo.blockSizeID = max4MB;
	190	+ prefs->frameInfo.blockMode = blockIndependent;
	191	+ prefs->frameInfo.contentChecksumFlag = contentChecksumEnabled;
105	192	}
106		- prefs->compressionLevel = NUM2INT(level);
107	193	}
108	194
	195	+static struct encoder *
	196	+getencoderp(VALUE enc)
	197	+{
	198	+ return getrefp(enc, &encoder_type);
	199	+}
	200	+
	201	+static struct encoder *
	202	+getencoder(VALUE enc)
	203	+{
	204	+ return getref(enc, &encoder_type);
	205	+}
	206	+
109	207	/*
110	208	* call-seq:
111		- * initialize(outport, level = 1, blocklinked: false, streamsum: true)
	209	+ * initialize(outport = "".b, level = 1, blocksize: nil, blocklink: false, checksum: true)
112	210	*/
113	211	static VALUE
114	212	fenc_init(int argc, VALUE argv[], VALUE enc)
115	213	{
116		- struct encoder *p = getref(enc, &encoder_type);
	214	+ struct encoder *p = getencoder(enc);
117	215	VALUE outport;
118	216	fenc_init_args(argc, argv, &outport, &p->prefs);
119	217

		@@ -126,6 +224,7 @@
126	224	rb_str_set_len(p->workbuf, s);
127	225	rb_funcall2(outport, id_op_lshift, 1, &p->workbuf);
128	226	p->outport = outport;
	227	+ rb_obj_infect(p->outport, enc);
129	228	return enc;
130	229	}
131	230

		@@ -141,7 +240,7 @@
141	240	size_t destsize = LZ4F_compressBound(srcsize, &p->prefs);
142	241	aux_str_reserve(p->workbuf, destsize);
143	242	char *destp = RSTRING_PTR(p->workbuf);
144		- size_t size = LZ4F_compressUpdate(p->encoder, destp, destsize, srcp, srcsize, opts);
	243	+ size_t size = aux_LZ4F_compressUpdate(p->encoder, destp, destsize, srcp, srcsize, opts);
145	244	aux_lz4f_check_error(size);
146	245	rb_str_set_len(p->workbuf, size);
147	246	rb_funcall2(p->outport, id_op_lshift, 1, &p->workbuf);

		@@ -156,9 +255,12 @@
156	255	static VALUE
157	256	fenc_write(int argc, VALUE argv[], VALUE enc)
158	257	{
159		- struct encoder *p = getref(enc, &encoder_type);
	258	+ struct encoder *p = getencoder(enc);
160	259	VALUE src;
161	260	rb_scan_args(argc, argv, "1", &src);
	261	+ rb_obj_infect(enc, src);
	262	+ rb_obj_infect(enc, p->workbuf);
	263	+ rb_obj_infect(p->outport, enc);
162	264	fenc_update(p, src, NULL);
163	265	return enc;
164	266	}

		@@ -166,21 +268,22 @@
166	268	static VALUE
167	269	fenc_push(VALUE enc, VALUE src)
168	270	{
169		- struct encoder *p = getref(enc, &encoder_type);
	271	+ struct encoder *p = getencoder(enc);
	272	+ rb_obj_infect(enc, src);
	273	+ rb_obj_infect(enc, p->workbuf);
	274	+ rb_obj_infect(p->outport, enc);
170	275	fenc_update(p, src, NULL);
171	276	return enc;
172	277	}
173	278
174	279	static VALUE
175		-fenc_flush(int argc, VALUE argv[], VALUE enc)
	280	+fenc_flush(VALUE enc)
176	281	{
177		- struct encoder *p = getref(enc, &encoder_type);
	282	+ struct encoder *p = getencoder(enc);
178	283	size_t destsize = AUX_LZ4F_BLOCK_SIZE_MAX + AUX_LZ4F_FINISH_SIZE;
179	284	aux_str_reserve(p->workbuf, destsize);
180		- //rb_str_locktmp(p->workbuf);
181	285	char *destp = RSTRING_PTR(p->workbuf);
182	286	size_t size = LZ4F_flush(p->encoder, destp, destsize, NULL);
183		- //rb_str_unlocktmp(p->workbuf);
184	287	aux_lz4f_check_error(size);
185	288	rb_str_set_len(p->workbuf, size);
186	289	rb_funcall2(p->outport, id_op_lshift, 1, &p->workbuf);

		@@ -191,13 +294,11 @@
191	294	static VALUE
192	295	fenc_close(VALUE enc)
193	296	{
194		- struct encoder *p = getref(enc, &encoder_type);
	297	+ struct encoder *p = getencoder(enc);
195	298	size_t destsize = AUX_LZ4F_BLOCK_SIZE_MAX + AUX_LZ4F_FINISH_SIZE;
196	299	aux_str_reserve(p->workbuf, destsize);
197		- //rb_str_locktmp(p->workbuf);
198	300	char *destp = RSTRING_PTR(p->workbuf);
199	301	size_t size = LZ4F_compressEnd(p->encoder, destp, destsize, NULL);
200		- //rb_str_unlocktmp(p->workbuf);
201	302	aux_lz4f_check_error(size);
202	303	rb_str_set_len(p->workbuf, size);
203	304	rb_funcall2(p->outport, id_op_lshift, 1, &p->workbuf);

		@@ -205,6 +306,65 @@
205	306	return enc;
206	307	}
207	308
	309	+static VALUE
	310	+fenc_getoutport(VALUE enc)
	311	+{
	312	+ return getencoder(enc)->outport;
	313	+}
	314	+
	315	+static VALUE
	316	+fenc_setoutport(VALUE enc, VALUE outport)
	317	+{
	318	+ return getencoder(enc)->outport = outport;
	319	+}
	320	+
	321	+static VALUE
	322	+fenc_prefs_level(VALUE enc)
	323	+{
	324	+ return INT2NUM(aux_frame_level(&getencoder(enc)->prefs));
	325	+}
	326	+
	327	+static int
	328	+fenc_blocksize(struct encoder *p)
	329	+{
	330	+ return aux_frame_blocksize(&p->prefs.frameInfo);
	331	+}
	332	+
	333	+static VALUE
	334	+fenc_prefs_blocksize(VALUE enc)
	335	+{
	336	+ return INT2NUM(fenc_blocksize(getencoder(enc)));
	337	+}
	338	+
	339	+static VALUE
	340	+fenc_prefs_blocklink(VALUE enc)
	341	+{
	342	+ return aux_frame_blocklink(&getencoder(enc)->prefs.frameInfo) ? Qtrue : Qfalse;
	343	+}
	344	+
	345	+static VALUE
	346	+fenc_prefs_checksum(VALUE enc)
	347	+{
	348	+ return aux_frame_checksum(&getencoder(enc)->prefs.frameInfo) ? Qtrue : Qfalse;
	349	+}
	350	+
	351	+static VALUE
	352	+fenc_inspect(VALUE enc)
	353	+{
	354	+ struct encoder *p = getencoderp(enc);
	355	+ if (p) {
	356	+ return rb_sprintf("#<%s:%p outport=#<%s:%p>, level=%d, blocksize=%d, blocklink=%s, checksum=%s>",
	357	+ rb_obj_classname(enc), (void *)enc,
	358	+ rb_obj_classname(p->outport), (void *)p->outport,
	359	+ p->prefs.compressionLevel, fenc_blocksize(p),
	360	+ aux_frame_blocklink(&p->prefs.frameInfo) ? "true" : "false",
	361	+ aux_frame_checksum(&p->prefs.frameInfo) ? "true" : "false");
	362	+ } else {
	363	+ return rb_sprintf("#<%s:%p INVALID REFERENCE>",
	364	+ rb_obj_classname(enc), (void *)enc);
	365	+ }
	366	+}
	367	+
208	368	/* class LZ4::Decoder */
209	369
210	370	struct decoder

		@@ -211,10 +371,11 @@
211	371	{
212	372	VALUE inport;
213	373	VALUE readbuf; /* read buffer from inport */
214		- VALUE blockbuf; /* decoded lz4 frame block buffer */
215		- VALUE predict; /* preset dictionary (OBSOLUTE) / / FIXME: DELETE ME */
216		- size_t readsize; /* readblocksize in initialize */
217		- size_t status; /* status code of LZ4F_decompress */
	374	+ char blockbuf; / decoded lz4 frame block buffer */
	375	+ const char blockend; / end of blockbuf */
	376	+ char *blockhead;
	377	+ const char *blocktail;
	378	+ size_t status; /* status code of LZ4F_decompress */
218	379	LZ4F_frameInfo_t info;
219	380	LZ4F_decompressionContext_t decoder;
220	381	};

		@@ -226,8 +387,6 @@
226	387	struct decoder *p = pp;
227	388	rb_gc_mark(p->inport);
228	389	rb_gc_mark(p->readbuf);
229		- rb_gc_mark(p->blockbuf);
230		- rb_gc_mark(p->predict);
231	390	}
232	391	}
233	392

		@@ -239,6 +398,9 @@
239	398	if (p->decoder) {
240	399	LZ4F_freeDecompressionContext(p->decoder);
241	400	}
	401	+ if (p->blockbuf) {
	402	+ free(p->blockbuf);
	403	+ }
242	404	}
243	405	}
244	406

		@@ -256,12 +418,22 @@
256	418	VALUE obj = TypedData_Make_Struct(mod, struct decoder, &decoder_type, p);
257	419	p->inport = Qnil;
258	420	p->readbuf = Qnil;
259		- p->blockbuf = Qnil;
260		- p->predict = Qnil;
261		- p->status = ~(size_t)0;
	421	+ p->status = 0;
262	422	return obj;
263	423	}
264	424
	425	+static struct decoder *
	426	+getdecoderp(VALUE dec)
	427	+{
	428	+ return getrefp(dec, &decoder_type);
	429	+}
	430	+
	431	+static struct decoder *
	432	+getdecoder(VALUE dec)
	433	+{
	434	+ return getref(dec, &decoder_type);
	435	+}
	436	+
265	437	static inline VALUE
266	438	aux_read(VALUE obj, size_t size, VALUE buf)
267	439	{

		@@ -272,7 +444,7 @@
272	444	} else {
273	445	//fprintf(stderr, "%s:%d:%s: buffer.size=%d\n", __FILE__, __LINE__, __func__, (int)RSTRING_LEN(buf));
274	446	if (RSTRING_LEN(buf) > size) {
275		- rb_raise(rb_eRuntimeError, "read buffer is too big (%d, but expected to %d)", (int)RSTRING_LEN(buf), (int)size);
	447	+ rb_raise(rb_eRuntimeError, "most read (%d, but expected to %d)", (int)RSTRING_LEN(buf), (int)size);
276	448	}
277	449	return buf;
278	450	}

		@@ -290,15 +462,14 @@
290	462	static VALUE
291	463	fdec_init(int argc, VALUE argv[], VALUE dec)
292	464	{
293		- struct decoder *p = getref(dec, &decoder_type);
294		- VALUE inport, readblocksize;
	465	+ struct decoder *p = getdecoder(dec);
	466	+ VALUE inport;
	467	+ //VALUE readblocksize;
295	468	rb_scan_args(argc, argv, "1", &inport);
296	469	LZ4F_errorCode_t err = LZ4F_createDecompressionContext(&p->decoder, LZ4F_VERSION);
297	470	aux_lz4f_check_error(err);
	471	+ rb_obj_infect(dec, inport);
298	472	p->inport = inport;
299		- //p->readsize = NIL_P(readblocksize) ? WORK_BUFFER_SIZE : NUM2INT(readblocksize);
300		- p->readsize = 0;
301		- p->blockbuf = rb_str_buf_new(AUX_LZ4F_BLOCK_SIZE_MAX);
302	473	p->readbuf = rb_str_buf_new(0);
303	474	char *readp;
304	475	size_t readsize;

		@@ -322,6 +493,10 @@
322	493	s = LZ4F_getFrameInfo(p->decoder, &p->info, NULL, &zero);
323	494	aux_lz4f_check_error(s);
324	495
	496	+ size_t size = 1 << (p->info.blockSizeID * 2 + 8);
	497	+ p->blockbuf = ALLOC_N(char, size);
	498	+ p->blockend = p->blockbuf + size;
	499	+
325	500	return dec;
326	501	}
327	502

		@@ -349,19 +524,15 @@
349	524	}
350	525	}
351	526
352		-static void
353		-fdec_read_fetch(char *blockp, size_t blocksize, struct decoder *p)
	527	+static size_t
	528	+fdec_read_fetch(VALUE dec, struct decoder *p)
354	529	{
355		- RSTRING_GETMEM(p->blockbuf, blockp, blocksize);
356		- if (*blocksize > 0) {
357		- return;
358		- }
359		-
360		- while (*blocksize <= 0 && p->status != 0) {
361		- aux_read(p->inport, p->status, p->readbuf);
	530	+ size_t blocksize = p->blocktail - p->blockhead;
	531	+ while (blocksize <= 0 && p->status != 0) {
	532	+ VALUE v = aux_read(p->inport, p->status, p->readbuf);
362	533	char *readp;
363	534	size_t readsize;
364		- aux_str_getmem(p->readbuf, &readp, &readsize);
	535	+ aux_str_getmem(v, &readp, &readsize);
365	536	if (!readp) {
366	537	rb_raise(eError,
367	538	"read error - encounted invalid EOF - #<%s:%p>",

		@@ -368,10 +539,15 @@
368	539	rb_obj_classname(p->inport), (void *)p->inport);
369	540	}
370	541
371		- *blocksize = rb_str_capacity(p->blockbuf);
372		- p->status = LZ4F_decompress(p->decoder, *blockp, blocksize, readp, &readsize, NULL);
	542	+ rb_obj_infect(dec, v);
	543	+ blocksize = p->blockend - p->blockbuf;
	544	+ p->status = LZ4F_decompress(p->decoder, p->blockbuf, &blocksize, readp, &readsize, NULL);
373	545	aux_lz4f_check_error(p->status);
	546	+ p->blockhead = p->blockbuf;
	547	+ p->blocktail = p->blockhead + blocksize;
374	548	}
	549	+
	550	+ return blocksize;
375	551	}
376	552
377	553	/*

		@@ -383,11 +559,12 @@
383	559	static VALUE
384	560	fdec_read(int argc, VALUE argv[], VALUE dec)
385	561	{
386		- struct decoder *p = getref(dec, &decoder_type);
	562	+ struct decoder *p = getdecoder(dec);
387	563	size_t size;
388	564	VALUE dest;
389	565	fdec_read_args(argc, argv, &size, &dest);
390	566	if (size == 0) {
	567	+ rb_obj_infect(dest, dec);
391	568	return dest;
392	569	}
393	570

		@@ -395,21 +572,17 @@
395	572	return Qnil;
396	573	}
397	574
398		- rb_str_modify(p->blockbuf);
399		-
400	575	do {
401		- char *blockp;
402		- size_t blocksize;
403		- fdec_read_fetch(&blockp, &blocksize, p);
	576	+ size_t blocksize = fdec_read_fetch(dec, p);
	577	+ rb_obj_infect(dest, dec);
404	578
405	579	if (size < blocksize) {
406		- rb_str_buf_cat(dest, blockp, size);
407		- rb_str_set_len(p->blockbuf, blocksize);
408		- aux_str_drop_bytes(p->blockbuf, size);
409		- size = 0;
	580	+ rb_str_buf_cat(dest, p->blockhead, size);
	581	+ p->blockhead += size;
	582	+ break;
410	583	} else {
411		- rb_str_buf_cat(dest, blockp, blocksize);
412		- rb_str_set_len(p->blockbuf, 0);
	584	+ rb_str_buf_cat(dest, p->blockhead, blocksize);
	585	+ p->blocktail = p->blockhead = NULL;
413	586	size -= blocksize;
414	587	}
415	588	} while (size > 0 && p->status != 0);

		@@ -417,10 +590,58 @@
417	590	return dest;
418	591	}
419	592
	593	+/*
	594	+ * call-seq:
	595	+ * getc -> String \| nil
	596	+ *
	597	+ * Read one byte character.
	598	+ */
420	599	static VALUE
	600	+fdec_getc(VALUE dec)
	601	+{
	602	+ struct decoder *p = getdecoder(dec);
	603	+
	604	+ for (;;) {
	605	+ if (p->status == 0) {
	606	+ return Qnil;
	607	+ }
	608	+ size_t blocksize = fdec_read_fetch(dec, p);
	609	+ if (blocksize != 0) {
	610	+ char ch = (uint8_t)*p->blockhead;
	611	+ p->blockhead ++;
	612	+ return rb_str_new(&ch, 1);
	613	+ }
	614	+ }
	615	+}
	616	+
	617	+/*
	618	+ * call-seq:
	619	+ * getbyte -> Integer \| nil
	620	+ *
	621	+ * Read one byte code integer.
	622	+ */
	623	+static VALUE
	624	+fdec_getbyte(VALUE dec)
	625	+{
	626	+ struct decoder *p = getdecoder(dec);
	627	+
	628	+ for (;;) {
	629	+ if (p->status == 0) {
	630	+ return Qnil;
	631	+ }
	632	+ size_t blocksize = fdec_read_fetch(dec, p);
	633	+ if (blocksize != 0) {
	634	+ int ch = (uint8_t)*p->blockhead;
	635	+ p->blockhead ++;
	636	+ return INT2FIX(ch);
	637	+ }
	638	+ }
	639	+}
	640	+
	641	+static VALUE
421	642	fdec_close(VALUE dec)
422	643	{
423		- struct decoder *p = getref(dec, &decoder_type);
	644	+ struct decoder *p = getdecoder(dec);
424	645	p->status = 0;
425	646	// TODO: destroy decoder
426	647	return dec;

		@@ -429,7 +650,7 @@
429	650	static VALUE
430	651	fdec_eof(VALUE dec)
431	652	{
432		- struct decoder *p = getref(dec, &decoder_type);
	653	+ struct decoder *p = getdecoder(dec);
433	654	if (p->status == 0) {
434	655	return Qtrue;
435	656	} else {

		@@ -437,6 +658,53 @@
437	658	}
438	659	}
439	660
	661	+static VALUE
	662	+fdec_inport(VALUE dec)
	663	+{
	664	+ return getdecoder(dec)->inport;
	665	+}
	666	+
	667	+static int
	668	+fdec_blocksize(struct decoder *p)
	669	+{
	670	+ return aux_frame_blocksize(&p->info);
	671	+}
	672	+
	673	+static VALUE
	674	+fdec_prefs_blocksize(VALUE dec)
	675	+{
	676	+ return INT2NUM(fdec_blocksize(getdecoder(dec)));
	677	+}
	678	+
	679	+static VALUE
	680	+fdec_prefs_blocklink(VALUE dec)
	681	+{
	682	+ return aux_frame_blocklink(&getdecoder(dec)->info) ? Qtrue : Qfalse;
	683	+}
	684	+
	685	+static VALUE
	686	+fdec_prefs_checksum(VALUE dec)
	687	+{
	688	+ return aux_frame_checksum(&getdecoder(dec)->info) ? Qtrue : Qfalse;
	689	+}
	690	+
	691	+static VALUE
	692	+fdec_inspect(VALUE dec)
	693	+{
	694	+ struct decoder *p = getdecoderp(dec);
	695	+ if (p) {
	696	+ return rb_sprintf("#<%s:%p inport=#<%s:%p>, blocksize=%d, blocklink=%s, checksum=%s>",
	697	+ rb_obj_classname(dec), (void *)dec,
	698	+ rb_obj_classname(p->inport), (void *)p->inport,
	699	+ fdec_blocksize(p),
	700	+ aux_frame_blocklink(&p->info) ? "true" : "false",
	701	+ aux_frame_checksum(&p->info) ? "true" : "false");
	702	+ } else {
	703	+ return rb_sprintf("#<%s:%p INVALID REFERENCE>",
	704	+ rb_obj_classname(dec), (void *)dec);
	705	+ }
	706	+}
	707	+
440	708	/* setup for LZ4::Encoder and LZ4::Decoder */
441	709
442	710	void

		@@ -450,14 +718,30 @@
450	718	rb_define_method(cEncoder, "initialize", RUBY_METHOD_FUNC(fenc_init), -1);
451	719	rb_define_method(cEncoder, "write", RUBY_METHOD_FUNC(fenc_write), -1);
452	720	rb_define_method(cEncoder, "<<", RUBY_METHOD_FUNC(fenc_push), 1);
453		- rb_define_method(cEncoder, "flush", RUBY_METHOD_FUNC(fenc_flush), -1);
	721	+ rb_define_method(cEncoder, "flush", RUBY_METHOD_FUNC(fenc_flush), 0);
454	722	rb_define_method(cEncoder, "close", RUBY_METHOD_FUNC(fenc_close), 0);
	723	+ rb_define_alias(cEncoder, "finish", "close");
	724	+ rb_define_method(cEncoder, "outport", RUBY_METHOD_FUNC(fenc_getoutport), 0);
	725	+ rb_define_method(cEncoder, "outport=", RUBY_METHOD_FUNC(fenc_setoutport), 1);
	726	+ rb_define_method(cEncoder, "prefs_level", RUBY_METHOD_FUNC(fenc_prefs_level), 0);
	727	+ rb_define_method(cEncoder, "prefs_blocksize", RUBY_METHOD_FUNC(fenc_prefs_blocksize), 0);
	728	+ rb_define_method(cEncoder, "prefs_blocklink", RUBY_METHOD_FUNC(fenc_prefs_blocklink), 0);
	729	+ rb_define_method(cEncoder, "prefs_checksum", RUBY_METHOD_FUNC(fenc_prefs_checksum), 0);
	730	+ rb_define_method(cEncoder, "inspect", RUBY_METHOD_FUNC(fenc_inspect), 0);
455	731
456	732	VALUE cDecoder = rb_define_class_under(mLZ4, "Decoder", rb_cObject);
457	733	rb_define_alloc_func(cDecoder, fdec_alloc);
458	734	rb_define_method(cDecoder, "initialize", RUBY_METHOD_FUNC(fdec_init), -1);
459	735	rb_define_method(cDecoder, "read", RUBY_METHOD_FUNC(fdec_read), -1);
	736	+ rb_define_method(cDecoder, "getc", RUBY_METHOD_FUNC(fdec_getc), 0);
	737	+ rb_define_method(cDecoder, "getbyte", RUBY_METHOD_FUNC(fdec_getbyte), 0);
460	738	rb_define_method(cDecoder, "close", RUBY_METHOD_FUNC(fdec_close), 0);
	739	+ rb_define_alias(cDecoder, "finish", "close");
461	740	rb_define_method(cDecoder, "eof", RUBY_METHOD_FUNC(fdec_eof), 0);
	741	+ rb_define_method(cDecoder, "inport", RUBY_METHOD_FUNC(fdec_inport), 0);
462	742	rb_define_alias(cDecoder, "eof?", "eof");
	743	+ rb_define_method(cDecoder, "prefs_blocksize", RUBY_METHOD_FUNC(fdec_prefs_blocksize), 0);
	744	+ rb_define_method(cDecoder, "prefs_blocklink", RUBY_METHOD_FUNC(fdec_prefs_blocklink), 0);
	745	+ rb_define_method(cDecoder, "prefs_checksum", RUBY_METHOD_FUNC(fdec_prefs_checksum), 0);
	746	+ rb_define_method(cDecoder, "inspect", RUBY_METHOD_FUNC(fdec_inspect), 0);
463	747	}

--- trunk/extlz4/ext/extconf.rb (revision 70)

+++ trunk/extlz4/ext/extconf.rb (revision 71)

		@@ -7,7 +7,11 @@
7	7
8	8	$srcs = Dir.glob(File.join(File.dirname(__FILE__).gsub(/[\[\{\?\]/, "[\\0]"), "{.,../contrib/}/*.c")).map { \|n\| File.basename n }
9	9	$VPATH << "$(srcdir)/../contrib/lz4"
10		-find_header "lz4.h", "$(srcdir)/../contrib/lz4"
11		-find_header "xxhash.h", "$(srcdir)/../contrib/lz4"
	10	+find_header "lz4.h", "$(srcdir)/../contrib/lz4" or abort 1
	11	+find_header "xxhash.h", "$(srcdir)/../contrib/lz4" or abort 1
12	12
	13	+if RbConfig::CONFIG["arch"] =~ /mingw/
	14	+ $LDFLAGS << " -static-libgcc"
	15	+end
	16	+
13	17	create_makefile("extlz4")

--- trunk/extlz4/ext/extlz4.c (revision 70)

+++ trunk/extlz4/ext/extlz4.c (revision 71)

		@@ -42,8 +42,6 @@
42	42	{
43	43	mLZ4 = rb_define_module("LZ4");
44	44
45		- rb_define_const(mLZ4, "LZ4", mLZ4);
46		-
47	45	/*
48	46	* Document-const: LZ4::LIBVERSION
49	47	*

		@@ -66,6 +64,6 @@
66	64
67	65	eError = rb_define_class_under(mLZ4, "Error", rb_eRuntimeError);
68	66
69		- extlz4_init_rawapi();
	67	+ extlz4_init_blockapi();
70	68	extlz4_init_frameapi();
71	69	}

--- trunk/extlz4/ext/extlz4.h (revision 70)

+++ trunk/extlz4/ext/extlz4.h (revision 71)

		@@ -8,7 +8,7 @@
8	8	extern VALUE mLZ4; /* module LZ4 */
9	9	extern VALUE eError; /* class LZ4::Error < ::RuntimeError */
10	10
11		-extern void extlz4_init_rawapi(void);
	11	+extern void extlz4_init_blockapi(void);
12	12	extern void extlz4_init_frameapi(void);
13	13
14	14	#define AUX_FUNCALL(RECV, METHOD, ...) \

--- trunk/extlz4/Rakefile (revision 70)

+++ trunk/extlz4/Rakefile (revision 71)

		@@ -2,7 +2,8 @@
2	2	require "rake/clean"
3	3
4	4	DOC = FileList["{README,LICENSE,CHANGELOG,Changelog,HISTORY}{,.ja}{,.txt,.rd,.rdoc,.md,.markdown}"] +
5		- FileList["ext/**/{README,LICENSE,CHANGELOG,Changelog,HISTORY}{,.ja}{,.txt,.rd,.rdoc,.md,.markdown}"]
	5	+ FileList["{contrib,ext}/**/{README,LICENSE,CHANGELOG,Changelog,HISTORY}{,.ja}{,.txt,.rd,.rdoc,.md,.markdown}"] +
	6	+ FileList["ext/*/.{c,C,cc,cxx,cpp,h,H,hh}"]
6	7	#EXT = FileList["ext/*/.{h,hh,c,cc,cpp,cxx}"] +
7	8	# FileList["ext/externals/*/"]
8	9	EXT = FileList["ext/*/"]

		@@ -9,6 +10,7 @@
9	10	BIN = FileList["bin/*"]
10	11	LIB = FileList["lib/*/.rb"]
11	12	SPEC = FileList["spec/*/"]
	13	+TEST = FileList["test/*/"]
12	14	EXAMPLE = FileList["examples/*/"]
13	15	GEMSTUB_SRC = "gemstub.rb"
14	16	RAKEFILE = [File.basename(__FILE__), GEMSTUB_SRC]

		@@ -25,7 +27,7 @@
25	27	GEMFILE = "#{GEMSTUB.name}-#{GEMSTUB.version}.gem"
26	28	GEMSPEC = "#{GEMSTUB.name}.gemspec"
27	29
28		-GEMSTUB.files += DOC + EXT + EXTCONF + BIN + LIB + SPEC + EXAMPLE + RAKEFILE + EXTRA
	30	+GEMSTUB.files += DOC + EXT + EXTCONF + BIN + LIB + SPEC + TEST + EXAMPLE + RAKEFILE + EXTRA
29	31	GEMSTUB.files.sort!
30	32	GEMSTUB.rdoc_options \|\|= %w(--charset UTF-8)
31	33	GEMSTUB.extra_rdoc_files += DOC + LIB + EXT.reject { \|n\| n.include?("/externals/") \|\| !%w(.h .hh .c .cc .cpp .cxx).include?(File.extname(n)) }

		@@ -82,7 +84,7 @@
82	84	desc "generate binary gemspec"
83	85	task "native-gemspec" => GEMSPEC_NATIVE
84	86
85		- file GEMFILE_NATIVE => DOC + EXT + EXTCONF + BIN + LIB + SPEC + EXAMPLE + SOFILES + RAKEFILE + [GEMSPEC_NATIVE] do
	87	+ file GEMFILE_NATIVE => DOC + EXT + EXTCONF + BIN + LIB + SPEC + TEST + EXAMPLE + SOFILES + RAKEFILE + [GEMSPEC_NATIVE] do
86	88	sh "gem build #{GEMSPEC_NATIVE}"
87	89	end
88	90

		@@ -123,8 +125,8 @@
123	125	task :all => GEMFILE
124	126
125	127	desc "generate local rdoc"
126		-task :rdoc => DOC + EXT + LIB do
127		- sh *(%w(rdoc) + GEMSTUB.rdoc_options + DOC + EXT + LIB)
	128	+task :rdoc => DOC + LIB do
	129	+ sh *(%w(rdoc) + GEMSTUB.rdoc_options + DOC + LIB)
128	130	end
129	131
130	132	desc "launch rspec"

		@@ -138,7 +140,7 @@
138	140	desc "generate gemspec"
139	141	task gemspec: GEMSPEC
140	142
141		-file GEMFILE => DOC + EXT + EXTCONF + BIN + LIB + SPEC + EXAMPLE + RAKEFILE + [GEMSPEC] do
	143	+file GEMFILE => DOC + EXT + EXTCONF + BIN + LIB + SPEC + TEST + EXAMPLE + RAKEFILE + [GEMSPEC] do
142	144	sh "gem build #{GEMSPEC}"
143	145	end
144	146

--- trunk/extlz4/lib/extlz4.rb (revision 70)

+++ trunk/extlz4/lib/extlz4.rb (revision 71)

		@@ -2,13 +2,16 @@
2	2
3	3	require "stringio"
4	4
5		-ver = RbConfig::CONFIG["ruby_version"]
	5	+ver = RUBY_VERSION.slice(/\d+\.\d+/)
6	6	soname = File.basename(__FILE__, ".rb") << ".so"
7	7	lib = File.join(File.dirname(__FILE__), ver, soname)
8		-if File.file?(lib)
	8	+case
	9	+when File.file?(lib)
9	10	require_relative File.join(ver, soname)
	11	+when File.file?(File.join(File.dirname(__FILE__), ver))
	12	+ require_relative soname
10	13	else
11		- require_relative soname
	14	+ require soname
12	15	end
13	16
14	17	require_relative "extlz4/version"

		@@ -17,6 +20,8 @@
17	20	# LZ4 data and streaming data processor.
18	21	#
19	22	module LZ4
	23	+ LZ4 = self
	24	+
20	25	#
21	26	# call-seq:
22	27	# decode_file(inpath, outpath) -> nil

		@@ -27,10 +32,10 @@
27	32	# Return nil always.
28	33	#
29	34	# [inpath]
30		- # Give input file path, or input io (liked) object its has ``read'' method.
	35	+ # Give input file path, or input IO (liked) object its has ``read'' method.
31	36	#
32	37	# [outpath]
33		- # Give output file path, or output io (liked) object its has ``<<'' method.
	38	+ # Give output file path, or output IO (liked) object its has ``<<'' method.
34	39	#
35	40	def self.decode_file(inpath, outpath)
36	41	open_file(inpath, "rb") do \|infile\|

		@@ -48,7 +53,7 @@
48	53
49	54	#
50	55	# call-seq:
51		- # encode_file(inpath, outpath, level = 1, opt = {}) -> nil
	56	+ # encode_file(inpath, outpath, level = 1, opts = {}) -> nil
52	57	#
53	58	# Encode regular file to lz4 file.
54	59	#

		@@ -56,21 +61,21 @@
56	61	# Return nil always.
57	62	#
58	63	# [inpath]
59		- # Give input file path, or input io (liked) object its has ``read'' method.
	64	+ # Give input file path, or input IO (liked) object its has ``read'' method.
60	65	#
61	66	# [outpath]
62		- # Give output file path, or output io (liked) object its has ``<<'' method.
	67	+ # Give output file path, or output IO (liked) object its has ``<<'' method.
63	68	#
64	69	# [level = 1 (Integer)]
65	70	# See LZ4.encode method.
66	71	#
67		- # [opt = {} (Hash)]
	72	+ # [opts = {} (Hash)]
68	73	# See LZ4.encode method.
69	74	#
70		- def self.encode_file(inpath, outpath, level = 1, **opt)
	75	+ def self.encode_file(inpath, outpath, args, *opts)
71	76	open_file(inpath, "rb") do \|infile\|
72	77	open_file(outpath, "wb") do \|outfile\|
73		- encode(outfile, level, **opt) do \|lz4\|
	78	+ encode(outfile, args, *opts) do \|lz4\|
74	79	inbuf = ""
75	80	slicesize = 1 << 20
76	81	lz4 << inbuf while infile.read(slicesize, inbuf)

		@@ -111,11 +116,11 @@
111	116	# encode(output_io, level = 1, opts = {}) -> stream encoder
112	117	# encode(output_io, level = 1, opts = {}) { \|stream_encoder\| ... } -> yield_status
113	118	#
114		- # Encode to LZ4 stream data. This is available streaming process, but posible sequential write only.
	119	+ # Encode to LZ4 Frame data. This is available streaming process.
115	120	#
116	121	# Created data is decodable by lz4-cli.
117	122	#
118		- # ==== 共通引数
	123	+ # ==== Common parameters
119	124	#
120	125	# [level = 1 (Integer)]
121	126	# 圧縮レベルを指定します。0 から 9 までの整数値が指定出来ます。

		@@ -124,20 +129,17 @@
124	129	#
125	130	# 4以上の値は、高効率圧縮器の圧縮レベルとして渡されます。
126	131	#
127		- # [block_dependency: false (true or false)]
	132	+ # [blocklink: false (true or false)]
128	133	# Enable or disable block dependency funcion. Default is false.
129	134	#
130		- # 真を与えた場合、ストリームの圧縮効率が向上しますが、特定のブロックのみを取り出すことが難しくなります。
	135	+ # 真を与えた場合、ストリームの圧縮効率が向上します。
131	136	#
132		- # [block_checksum: false (true or false)]
133		- # ブロックごとのチェックサム (XXhash32) の有効・無効を切り替えます。
134		- #
135		- # [stream_checksum: true (true or false)]
	137	+ # [checksum: true (true or false)]
136	138	# ストリーム全体のチェックサム (XXhash32) の有効・無効を切り替えます。
137	139	#
138	140	# ==== encode(source_string, level = 1, opts = {}) -> encoded_data
139	141	#
140		- # Basic stream encode method.
	142	+ # Basic encode method.
141	143	#
142	144	# [RETURN (String)]
143	145	# Encoded data as LZ4 stream

		@@ -147,19 +149,20 @@
147	149	#
148	150	# 文字符号情報は無視されて純粋なバイナリデータ列として処理されます。
149	151	#
150		- # ==== encode(output_io, level = 1, opts = {}) -> stream_encoder
	152	+ # ==== encode(output_io, level = 1, opts = {}) -> encoder
151	153	#
152		- # Available LZ4 stream encode.
	154	+ # Available streaming LZ4 Frame encode.
153	155	#
154	156	# Write to encoder for data encoding.
155	157	#
156		- # After finished encode process, you must call +StreamEncoder#close+.
	158	+ # After finished encode process, you must call Encoder#close.
157	159	#
158		- # Return stream encoder if given an IO object (or psudo-object).
	160	+ # Return stream encoder if given an IO (liked) object.
159	161	#
160		- # この圧縮器に『書き込む』ことでデータは圧縮されます。圧縮処理を完了するときには #close を呼び出す必要があります。
	162	+ # この圧縮器に『書き込む』ことでデータは圧縮されます。
	163	+ # 圧縮処理を完了するときには #close を呼び出す必要があります。
161	164	#
162		- # [RETURN (LZ4::StreamEncoder)]
	165	+ # [RETURN (LZ4::Encoder)]
163	166	#
164	167	# [output_io (IO)]
165	168	# LZ4 ストリームの出力先を指定します。IO#<< と同等の機能を持つオブジェクトである必要があります。

		@@ -166,7 +169,7 @@
166	169	#
167	170	# 一例を挙げると、IO、StringIO、Array などのインスタンスが当てはまります。
168	171	#
169		- # ==== encode(output_io, level = 1, opts = {}) { \|stream_encoder\| ... } -> yield_status
	172	+ # ==== encode(output_io, level = 1, opts = {}) { \|encoder\| ... } -> yield_status
170	173	#
171	174	# IO オブジェクトとともにブロックを渡した場合、ブロック引数として圧縮器が渡されます。この場合は #close を呼び出す必要がありません。
172	175	#

		@@ -173,16 +176,16 @@
173	176	# [RETURN]
174	177	# return value of given block
175	178	#
176		- # [YIELD (stream_encoder)]
	179	+ # [YIELD (encoder)]
177	180	#
178	181	# [YIELDRETURN]
179	182	# return as method return value
180	183	#
181		- # ==== example: directly stream encode
	184	+ # ==== example: directly encode
182	185	#
183	186	# LZ4.encode("abcdefghijklmn") # => Encoded LZ4 stream data (string object)
184	187	#
185		- # ==== example: stream encode with block
	188	+ # ==== example: streaming encode with block
186	189	#
187	190	# この用例は、encode_file の実装とほぼ同じです。丸写しで利用するよりは encode_file の利用を推奨します。
188	191	#

		@@ -195,22 +198,23 @@
195	198	# end
196	199	# end
197	200	#
198		-
199		- def self.encode(obj, level = 1, **opts)
200		- if obj.kind_of?(String)
201		- lz4 = LZ4::Encoder.new(out = "".force_encoding(Encoding::BINARY), level, **opts)
202		- lz4 << obj
203		- lz4.close
204		- out
205		- else
206		- lz4 = LZ4::Encoder.new(obj, level, **opts)
	201	+ def self.encode(args, *opts)
	202	+ if args.empty? \|\| !args[0].kind_of?(String)
	203	+ lz4 = LZ4::Encoder.new(args, *opts)
207	204	return lz4 unless block_given?
208	205	begin
209	206	yield(lz4)
210		- obj
	207	+ lz4.outport
211	208	ensure
212	209	lz4.close
213	210	end
	211	+ else
	212	+ obj = args.shift
	213	+ outport = "".force_encoding(Encoding::BINARY)
	214	+ lz4 = LZ4::Encoder.new(outport, args, *opts)
	215	+ lz4 << obj
	216	+ lz4.close
	217	+ outport
214	218	end
215	219	end
216	220

		@@ -217,10 +221,10 @@
217	221	#
218	222	# call-seq:
219	223	# decode(encoded_data_string) -> decoded data
220		- # decode(input_io) -> stream_decoder
221		- # decode(input_io) { \|stream_decoder\| ... } -> yield_status
	224	+ # decode(input_io) -> decoder
	225	+ # decode(input_io) { \|decoder\| ... } -> yield_status
222	226	#
223		- # Decode LZ4 stream data. This is available streaming process.
	227	+ # Decode LZ4 Frame data. This is available streaming process.
224	228	#
225	229	# ==== decode(encoded_data_string)
226	230	#

		@@ -229,35 +233,29 @@
229	233	#
230	234	# ==== decode(input_io)
231	235	#
232		- # [RETURN (LZ4::StreamDecoder)]
	236	+ # [RETURN (LZ4::Decoder)]
233	237	# ストリーム展開オブジェクトです。簡素な機能の読み込み専用IOオブジェクトとして扱うことが出来ます。
234	238	#
235		- # stream_decoder は GC によって開放処理が行われますが、利用しなくなった時点で利用者が明示的に close を呼び出すことが望まれます。
	239	+ # decoder は GC によって開放処理が行われますが、利用しなくなった時点で利用者が明示的に close を呼び出すことが望まれます。
236	240	#
237	241	# [input_io (IO)]
238	242	# This is IO like object. Need read method. 'extlz4' is call as <tt>read(size, buf)</tt> style.
239	243	#
240		- # ==== decode(input_io) { \|stream_decoder\| ... }
	244	+ # ==== decode(input_io) { \|decoder\| ... }
241	245	#
242	246	# [RETURN]
243	247	# returned value from given block
244	248	#
245		- # [YIELD (stream_decoder)]
	249	+ # [YIELD (decoder)]
246	250	# ブロックなしで与えた場合の戻り値と等価です。
247	251	#
248	252	# ただしこちらはブロックを抜けたらすぐに開放処理が実施されます。利用者が明示的に close を呼んだり、GC されるのを待ったりせずに行われると言うことです。
249	253	#
250		- # ==== note
251		- #
252		- # Current implementation is possible 'sequential read' only.
253		- #
254		- # 'read behind' or 'random read' are not available.
255		- #
256	254	# ==== example: directly decode
257	255	#
258		- # LZ4.decode(lz4_stream_encoded_string) # => decoded binary string
	256	+ # LZ4.decode(lz4_encoded_string) # => decoded binary string
259	257	#
260		- # ==== example: stream decode
	258	+ # ==== example: streaming decode
261	259	#
262	260	# File.open("sample.lz4", "rb") do \|fd\|
263	261	# LZ4.decode(fd) do \|lz4dec\|

		@@ -267,15 +265,15 @@
267	265	# end
268	266	# end
269	267	#
270		- def self.decode(obj, *opts)
	268	+ def self.decode(obj, *args)
271	269	if obj.kind_of?(String)
272		- lz4 = Decoder.new(StringIO.new(obj), *opts)
	270	+ lz4 = Decoder.new(StringIO.new(obj), *args)
273	271	dest = lz4.read
274	272	lz4.close
275	273	return (dest \|\| "".b)
276	274	end
277	275
278		- lz4 = Decoder.new(obj, *opts)
	276	+ lz4 = Decoder.new(obj, *args)
279	277	return lz4 unless block_given?
280	278
281	279	begin

		@@ -285,19 +283,19 @@
285	283	end
286	284	end
287	285
288		- def self.raw_encode(*args)
289		- RawEncoder.encode(*args)
	286	+ def self.block_encode(*args)
	287	+ BlockEncoder.encode(*args)
290	288	end
291	289
292		- def self.raw_decode(*args)
293		- RawDecoder.decode(*args)
	290	+ def self.block_decode(*args)
	291	+ BlockDecoder.decode(*args)
294	292	end
295	293
296	294	#
297		- # Call LZ4::RawEncoder.new.
	295	+ # Call LZ4::BlockEncoder.new.
298	296	#
299		- def self.raw_stream_encode(*args)
300		- lz4 = RawEncoder.new(*args)
	297	+ def self.block_stream_encode(*args)
	298	+ lz4 = BlockEncoder.new(*args)
301	299	if block_given?
302	300	yield(lz4)
303	301	else

		@@ -306,10 +304,10 @@
306	304	end
307	305
308	306	#
309		- # Call LZ4::RawDecoder.new.
	307	+ # Call LZ4::BlockDecoder.new.
310	308	#
311		- def self.raw_stream_decode(*args)
312		- lz4 = RawDecoder.new(*args)
	309	+ def self.block_stream_decode(*args)
	310	+ lz4 = BlockDecoder.new(*args)
313	311	if block_given?
314	312	yield(lz4)
315	313	else

		@@ -321,11 +319,13 @@
321	319	alias compress encode
322	320	alias decompress decode
323	321	alias uncompress decode
324		- alias raw_compress raw_encode
325		- alias raw_decompress raw_decode
326		- alias raw_uncompress raw_decode
327		- alias raw_stream_compress raw_stream_encode
328		- alias raw_stream_decompress raw_stream_decode
329		- alias raw_stream_uncompress raw_stream_decode
	322	+ alias block_compress block_encode
	323	+ alias block_decompress block_decode
	324	+ alias block_uncompress block_decode
	325	+ alias block_stream_compress block_stream_encode
	326	+ alias block_stream_decompress block_stream_decode
	327	+ alias block_stream_uncompress block_stream_decode
330	328	end
331	329	end
	330	+
	331	+require_relative "extlz4/compat"

--- trunk/extlz4/lib/extlz4/oldstream.rb (revision 70)

+++ trunk/extlz4/lib/extlz4/oldstream.rb (revision 71)

		@@ -8,8 +8,11 @@
8	8	# dearblue <dearblue@users.sourceforce.jp>
9	9	#
10	10
11		-require "extlz4"
	11	+require_relative "../extlz4"
12	12	require "stringio"
	13	+
	14	+require "rubygems"
	15	+gem "xxhash", "~> 0.3"
13	16	require "xxhash"
14	17
15	18	module LZ4

		@@ -178,7 +181,7 @@
178	181	header << desc
179	182	header << [streamsize].pack("Q<") if streamsize
180	183	header << [predictid].pack("V") if predictid
181		- header << [XXhash32.digest(desc) >> 8].pack("C")
	184	+ header << [XXhash.xxh32(desc) >> 8].pack("C")
182	185	end
183	186	end
184	187

		@@ -228,7 +231,7 @@
228	231	def initialize(io, level, blocksize, block_dependency,
229	232	block_checksum, stream_checksum)
230	233	@block_checksum = !!block_checksum
231		- @stream_checksum = XXhash32.new if stream_checksum
	234	+ @stream_checksum = XXhash::XXhashInternal::StreamingHash32.new(0) if stream_checksum
232	235
233	236	@blocksize = BLOCK_MAXIMUM_SIZES[blocksize]
234	237	raise ArgumentError, "wrong blocksize (#{blocksize})" unless @blocksize

		@@ -256,8 +259,8 @@
256	259	desc = [sd, bd].pack("CC")
257	260	header << desc
258	261	# TODO: header << [stream_size].pack("Q<") if stream_size
259		- # TODO: header << [XXhash32.digest(predict)].pack("V") if predict # preset dictionary
260		- header << [XXhash32.digest(desc) >> 8].pack("C")
	262	+ # TODO: header << [XXhash.xxh32(predict)].pack("V") if predict # preset dictionary
	263	+ header << [XXhash.xxh32(desc) >> 8].pack("C")
261	264	@io << header
262	265	end
263	266

		@@ -312,10 +315,10 @@
312	315	def get_encoder(level, block_dependency)
313	316	workencbuf = "".force_encoding(Encoding::BINARY)
314	317	if block_dependency
315		- streamencoder = LZ4::RawStreamEncoder.new(@blocksize, level)
316		- ->(src) { streamencoder.update(level, src, workencbuf) }
	318	+ streamencoder = LZ4::BlockEncoder.new(level)
	319	+ ->(src) { streamencoder.update(src, workencbuf) }
317	320	else
318		- ->(src) { LZ4.raw_encode(level, src, workencbuf) }
	321	+ ->(src) { LZ4.block_encode(level, src, workencbuf) }
319	322	end
320	323	end
321	324

		@@ -333,7 +336,7 @@
333	336	end
334	337
335	338	if @block_checksum
336		- @io << [XXhash32.digest(w)].pack("V")
	339	+ @io << [XXhash.xxh32(w)].pack("V")
337	340	end
338	341	@buf.clear
339	342	end

		@@ -381,9 +384,9 @@
381	384	headerchecksum = io.getbyte
382	385
383	386	if @blockindependence
384		- @decoder = LZ4.method(:raw_decode)
	387	+ @decoder = LZ4.method(:block_decode)
385	388	else
386		- @decoder = LZ4::RawStreamDecoder.new.method(:update)
	389	+ @decoder = LZ4::BlockDecoder.new.method(:update)
387	390	end
388	391	when MAGIC_NUMBER_LEGACY
389	392	@version = -1

		@@ -393,7 +396,7 @@
393	396	@blockmaximum = 1 << 23 # 8 MiB
394	397	@streamsize = nil
395	398	@presetdict = nil
396		- @decoder = LZ4.method(:raw_decode)
	399	+ @decoder = LZ4.method(:block_decode)
397	400	else
398	401	raise Error, "stream header error - wrong magic number"
399	402	end

--- trunk/extlz4/lib/extlz4/compat.rb (nonexistent)

+++ trunk/extlz4/lib/extlz4/compat.rb (revision 71)

		@@ -0,0 +1,12 @@
	1	+
	2	+module LZ4
	3	+ class << self
	4	+ alias raw_encode block_encode
	5	+ alias raw_decode block_decode
	6	+ alias raw_stream_encode block_stream_encode
	7	+ alias raw_stream_decode block_stream_decode
	8	+ end
	9	+
	10	+ RawStreamEncoder = BlockEncoder
	11	+ RawStreamDecoder = BlockDecoder
	12	+end

--- trunk/extlz4/HISTORY.ja.md (revision 70)

+++ trunk/extlz4/HISTORY.ja.md (revision 71)

		@@ -1,25 +1,39 @@
1	1
2		-# extlz4-0.2 (2015-03-26)
	2	+# extlz4-0.2 (2015-04-19)
3	3
4		-## ファイル構成の変更
	4	+## いくつかの名称の変更
5	5
6		-* ext/: ext/extlz4.c を複数のファイルに分割しました。
	6	+ * ストリームをフレーム (frame) に変更しました。
	7	+ * これまで extlz4 において raw\\\* と呼んできた名称を block\\\* に変更しました。
7	8
8	9	## LZ4 ストリームの独自実装から LZ4 Frame API への移行
9	10
10		-* lib/extlz4.rb: LZ4.encode、LZ4.decode の引数は互換性を失いました。
11		-* lib/extlz4/oldstream.rb: 独自実装版は LZ4::StreamEncoder、LZ4::StreamDecoder のまま残されました。
12		- これ以上保守されませんし、将来的にこのクラスは廃止されます。
13		- 利用する場合は ``require "extlz4/oldstream"`` とする必要があります。
	11	+ * LZ4.encode、LZ4.decode の引数は互換性を失いました。
	12	+ * LZ4 Frame API による圧縮・伸長処理を行うためのクラスは
	13	+ LZ4::Encoder と LZ4::Decoder として利用できます。
	14	+ * 独自実装版は LZ4::StreamEncoder、LZ4::StreamDecoder のまま残されました。
	15	+ * *これらのクラスは将来的に廃止される予定です。*
	16	+ * 利用する場合は ``require "extlz4/oldstream"`` とする必要があります。
	17	+ * ruby gems の xxhash-0.3 を必要とします。
	18	+ * 以前の LZ4.encode は LZ4.encode\_old、LZ4.decode は LZ4.decode\_old
	19	+ として利用できます。
14	20
15		-## LZ4 streaming API に対する更新
	21	+## 新しい LZ4 Block Streaming API への移行
16	22
17		-* ext/: ``LZ4_create()`` 系から ``LZ4_createStream()`` 系の API に移行しました。
18		-* ext/: ``LZ4_decompress_safe_withPrefix64k()`` から ``LZ4_createStreamDecode()`` 系の API に移行しました。
19		-* ext/: ``LZ4::RawStreamEncoder`` が ``LZ4::RawEncoder`` になりました。
20		-* ext/: ``LZ4::RawStreamDecoder`` が ``LZ4::RawDecoder`` になりました。
	23	+ * ``LZ4_create()`` 系から ``LZ4_createStream()`` 系の API に移行しました。
	24	+ * ``LZ4_decompress_safe_withPrefix64k()`` から ``LZ4_createStreamDecode()`` 系の API に移行しました。
	25	+ * LZ4::RawStreamEncoder が LZ4::BlockEncoder になりました。
	26	+ * LZ4::RawStreamDecoder が LZ4::BlockDecoder になりました。
21	27
	28	+## セーフレベルの確認処理を削除
22	29
	30	+ * セーフレベルの確認処理を削除しました。
	31	+
	32	+ 今まではセーフレベルが4以上の場合に汚染状態を移す必要のある場合は、
	33	+ SecurityError 例外を発生させていましたが、この方針を変更して常に
	34	+ 汚染状態を伝搬させるだけの処理にしました。
	35	+
	36	+
23	37	# extlz4-0.1.1 (2014-06-01)
24	38
25	39	## 不具合の修正

		@@ -34,7 +48,7 @@
34	48
35	49	* lib/extlz4.rb: ブロック依存ストリーム生成の場合、高効率圧縮時に圧縮レベルが常に規定値になっていましたが、これを変動するように修正しました。
36	50
37		-* lib/extlz4.rb (`LZ4::StreamEncoder#initialize`): `raw_encode` / `RawStreamEncoder#update` に渡す `level` の値が [nil, 0 .. 16] になるように修正しました。
	51	+* lib/extlz4.rb (`LZ4::StreamEncoder#initialize`): `block_encode` / `RawStreamEncoder#update` に渡す `level` の値が [nil, 0 .. 16] になるように修正しました。
38	52
39	53	* bin/extlz4: lz4 ストリーム検査の時、標準入力を利用した場合でも『-f』スイッチが必要となっていましたが、これを不要とするように修正しました。
40	54

旧リポジトリブラウザで表示