1package Encode::Encoding; 2# Base class for classes which implement encodings 3use strict; 4our $VERSION = do { my @r = (q$Revision: 2.2 $ =~ /\d+/g); sprintf "%d."."%02d" x $#r, @r }; 5 6require Encode; 7 8sub DEBUG { 0 } 9sub Define 10{ 11 my $obj = shift; 12 my $canonical = shift; 13 $obj = bless { Name => $canonical },$obj unless ref $obj; 14 # warn "$canonical => $obj\n"; 15 Encode::define_encoding($obj, $canonical, @_); 16} 17 18sub name { return shift->{'Name'} } 19 20# sub renew { return $_[0] } 21 22sub renew { 23 my $self = shift; 24 my $clone = bless { %$self } => ref($self); 25 $clone->{renewed}++; # so the caller can see it 26 DEBUG and warn $clone->{renewed}; 27 return $clone; 28} 29 30sub renewed{ return $_[0]->{renewed} || 0 } 31 32*new_sequence = \&renew; 33 34sub needs_lines { 0 }; 35 36sub perlio_ok { 37 eval{ require PerlIO::encoding }; 38 return $@ ? 0 : 1; 39} 40 41# (Temporary|legacy) methods 42 43sub toUnicode { shift->decode(@_) } 44sub fromUnicode { shift->encode(@_) } 45 46# 47# Needs to be overloaded or just croak 48# 49 50sub encode { 51 require Carp; 52 my $obj = shift; 53 my $class = ref($obj) ? ref($obj) : $obj; 54 Carp::croak($class . "->encode() not defined!"); 55} 56 57sub decode{ 58 require Carp; 59 my $obj = shift; 60 my $class = ref($obj) ? ref($obj) : $obj; 61 Carp::croak($class . "->encode() not defined!"); 62} 63 64sub DESTROY {} 65 661; 67__END__ 68 69=head1 NAME 70 71Encode::Encoding - Encode Implementation Base Class 72 73=head1 SYNOPSIS 74 75 package Encode::MyEncoding; 76 use base qw(Encode::Encoding); 77 78 __PACKAGE__->Define(qw(myCanonical myAlias)); 79 80=head1 DESCRIPTION 81 82As mentioned in L<Encode>, encodings are (in the current 83implementation at least) defined as objects. The mapping of encoding 84name to object is via the C<%Encode::Encoding> hash. Though you can 85directly manipulate this hash, it is strongly encouraged to use this 86base class module and add encode() and decode() methods. 87 88=head2 Methods you should implement 89 90You are strongly encouraged to implement methods below, at least 91either encode() or decode(). 92 93=over 4 94 95=item -E<gt>encode($string [,$check]) 96 97MUST return the octet sequence representing I<$string>. 98 99=over 2 100 101=item * 102 103If I<$check> is true, it SHOULD modify I<$string> in place to remove 104the converted part (i.e. the whole string unless there is an error). 105If perlio_ok() is true, SHOULD becomes MUST. 106 107=item * 108 109If an error occurs, it SHOULD return the octet sequence for the 110fragment of string that has been converted and modify $string in-place 111to remove the converted part leaving it starting with the problem 112fragment. If perlio_ok() is true, SHOULD becomes MUST. 113 114=item * 115 116If I<$check> is is false then C<encode> MUST make a "best effort" to 117convert the string - for example, by using a replacement character. 118 119=back 120 121=item -E<gt>decode($octets [,$check]) 122 123MUST return the string that I<$octets> represents. 124 125=over 2 126 127=item * 128 129If I<$check> is true, it SHOULD modify I<$octets> in place to remove 130the converted part (i.e. the whole sequence unless there is an 131error). If perlio_ok() is true, SHOULD becomes MUST. 132 133=item * 134 135If an error occurs, it SHOULD return the fragment of string that has 136been converted and modify $octets in-place to remove the converted 137part leaving it starting with the problem fragment. If perlio_ok() is 138true, SHOULD becomes MUST. 139 140=item * 141 142If I<$check> is false then C<decode> should make a "best effort" to 143convert the string - for example by using Unicode's "\x{FFFD}" as a 144replacement character. 145 146=back 147 148=back 149 150If you want your encoding to work with L<encoding> pragma, you should 151also implement the method below. 152 153=over 4 154 155=item -E<gt>cat_decode($destination, $octets, $offset, $terminator [,$check]) 156 157MUST decode I<$octets> with I<$offset> and concatenate it to I<$destination>. 158Decoding will terminate when $terminator (a string) appears in output. 159I<$offset> will be modified to the last $octets position at end of decode. 160Returns true if $terminator appears output, else returns false. 161 162=back 163 164=head2 Other methods defined in Encode::Encodings 165 166You do not have to override methods shown below unless you have to. 167 168=over 4 169 170=item -E<gt>name 171 172Predefined As: 173 174 sub name { return shift->{'Name'} } 175 176MUST return the string representing the canonical name of the encoding. 177 178=item -E<gt>renew 179 180Predefined As: 181 182 sub renew { 183 my $self = shift; 184 my $clone = bless { %$self } => ref($self); 185 $clone->{renewed}++; 186 return $clone; 187 } 188 189This method reconstructs the encoding object if necessary. If you need 190to store the state during encoding, this is where you clone your object. 191 192PerlIO ALWAYS calls this method to make sure it has its own private 193encoding object. 194 195=item -E<gt>renewed 196 197Predefined As: 198 199 sub renewed { $_[0]->{renewed} || 0 } 200 201Tells whether the object is renewed (and how many times). Some 202modules emit C<Use of uninitialized value in null operation> warning 203unless the value is numeric so return 0 for false. 204 205=item -E<gt>perlio_ok() 206 207Predefined As: 208 209 sub perlio_ok { 210 eval{ require PerlIO::encoding }; 211 return $@ ? 0 : 1; 212 } 213 214If your encoding does not support PerlIO for some reasons, just; 215 216 sub perlio_ok { 0 } 217 218=item -E<gt>needs_lines() 219 220Predefined As: 221 222 sub needs_lines { 0 }; 223 224If your encoding can work with PerlIO but needs line buffering, you 225MUST define this method so it returns true. 7bit ISO-2022 encodings 226are one example that needs this. When this method is missing, false 227is assumed. 228 229=back 230 231=head2 Example: Encode::ROT13 232 233 package Encode::ROT13; 234 use strict; 235 use base qw(Encode::Encoding); 236 237 __PACKAGE__->Define('rot13'); 238 239 sub encode($$;$){ 240 my ($obj, $str, $chk) = @_; 241 $str =~ tr/A-Za-z/N-ZA-Mn-za-m/; 242 $_[1] = '' if $chk; # this is what in-place edit means 243 return $str; 244 } 245 246 # Jr pna or ynml yvxr guvf; 247 *decode = \&encode; 248 249 1; 250 251=head1 Why the heck Encode API is different? 252 253It should be noted that the I<$check> behaviour is different from the 254outer public API. The logic is that the "unchecked" case is useful 255when the encoding is part of a stream which may be reporting errors 256(e.g. STDERR). In such cases, it is desirable to get everything 257through somehow without causing additional errors which obscure the 258original one. Also, the encoding is best placed to know what the 259correct replacement character is, so if that is the desired behaviour 260then letting low level code do it is the most efficient. 261 262By contrast, if I<$check> is true, the scheme above allows the 263encoding to do as much as it can and tell the layer above how much 264that was. What is lacking at present is a mechanism to report what 265went wrong. The most likely interface will be an additional method 266call to the object, or perhaps (to avoid forcing per-stream objects 267on otherwise stateless encodings) an additional parameter. 268 269It is also highly desirable that encoding classes inherit from 270C<Encode::Encoding> as a base class. This allows that class to define 271additional behaviour for all encoding objects. 272 273 package Encode::MyEncoding; 274 use base qw(Encode::Encoding); 275 276 __PACKAGE__->Define(qw(myCanonical myAlias)); 277 278to create an object with C<< bless {Name => ...}, $class >>, and call 279define_encoding. They inherit their C<name> method from 280C<Encode::Encoding>. 281 282=head2 Compiled Encodings 283 284For the sake of speed and efficiency, most of the encodings are now 285supported via a I<compiled form>: XS modules generated from UCM 286files. Encode provides the enc2xs tool to achieve that. Please see 287L<enc2xs> for more details. 288 289=head1 SEE ALSO 290 291L<perlmod>, L<enc2xs> 292 293=begin future 294 295=over 4 296 297=item Scheme 1 298 299The fixup routine gets passed the remaining fragment of string being 300processed. It modifies it in place to remove bytes/characters it can 301understand and returns a string used to represent them. For example: 302 303 sub fixup { 304 my $ch = substr($_[0],0,1,''); 305 return sprintf("\x{%02X}",ord($ch); 306 } 307 308This scheme is close to how the underlying C code for Encode works, 309but gives the fixup routine very little context. 310 311=item Scheme 2 312 313The fixup routine gets passed the original string, an index into 314it of the problem area, and the output string so far. It appends 315what it wants to the output string and returns a new index into the 316original string. For example: 317 318 sub fixup { 319 # my ($s,$i,$d) = @_; 320 my $ch = substr($_[0],$_[1],1); 321 $_[2] .= sprintf("\x{%02X}",ord($ch); 322 return $_[1]+1; 323 } 324 325This scheme gives maximal control to the fixup routine but is more 326complicated to code, and may require that the internals of Encode be tweaked to 327keep the original string intact. 328 329=item Other Schemes 330 331Hybrids of the above. 332 333Multiple return values rather than in-place modifications. 334 335Index into the string could be C<pos($str)> allowing C<s/\G...//>. 336 337=back 338 339=end future 340 341=cut 342