Class: YARD::Parser::SourceParser
- Inherits:
-
Object
- Object
- YARD::Parser::SourceParser
- Defined in:
- lib/yard/parser/source_parser.rb
Overview
Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.
Custom Parsers
SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call SourceParser.register_parser_type
Constant Summary
- SHEBANG_LINE =
/\A\s*#!\S+/
- ENCODING_LINE =
/\A(?:\s*#*!.*\r?\n)?\s*(?:#+|\/\*+|\/\/+).*coding\s*[:=]{1,2}\s*([a-z\d_\-]+)/i
- ENCODING_BYTE_ORDER_MARKS =
Byte order marks for various encodings
{ 'utf-8' => "\xEF\xBB\xBF", # Not yet supported #'utf-16be' => "\xFE\xFF", #'utf-16le' => "\xFF\xFE", #'utf-32be' => "\x00\x00\xFF\xFE", #'utf-32le' => "\xFF\xFE", }
Parser Callbacks (collapse)
-
- (String) contents
readonly
The contents of the file to be parsed.
-
- (String) file
The filename being parsed by the parser.
-
- (OpenStruct) globals
readonly
An open struct containing arbitrary global state shared between files and handlers.
-
- (Symbol) parser_type
readonly
The parser type associated with the parser instance.
Class Attribute Summary (collapse)
-
+ (Symbol) parser_type
The default parser type (defaults to :ruby).
- + (Object) parser_type_extensions writeonly
- + (Object) parser_types writeonly
Parser Callbacks (collapse)
-
+ (Proc) after_parse_file {|parser| ... }
Registers a callback to be called after an individual file is parsed.
-
+ (Array<Proc>) after_parse_file_callbacks
The list of callbacks to be called after parsing a file.
-
+ (Proc) after_parse_list {|files, globals| ... }
Registers a callback to be called after a list of files is parsed via SourceParser.parse.
-
+ (Array<Proc>) after_parse_list_callbacks
The list of callbacks to be called after parsing a list of files.
-
+ (Proc) before_parse_file {|parser| ... }
Registers a callback to be called before an individual file is parsed.
-
+ (Array<Proc>) before_parse_file_callbacks
The list of callbacks to be called before parsing a file.
-
+ (Proc) before_parse_list {|files, globals| ... }
Registers a callback to be called before a list of files is parsed via SourceParser.parse.
-
+ (Array<Proc>) before_parse_list_callbacks
The list of callbacks to be called before parsing a list of files.
-
- (SourceParser) initialize(parser_type = SourceParser.parser_type, globals = nil)
constructor
Creates a new parser object for code parsing with a specific parser type.
-
- (Object?) parse(content = __FILE__)
The main parser method.
-
- (Array) tokenize(content)
Tokenizes but does not parse the block of code using the current #parser_type.
Class Method Summary (collapse)
-
+ (void) parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
Parses a path or set of paths.
-
+ (Object) parse_string(content, ptype = parser_type)
Parses a string
content
. -
+ (Symbol) parser_type_for_extension(extension)
Finds a parser type that is registered for the extension.
-
+ (void) register_parser_type(type, parser_klass, extensions = nil)
Registers a new parser type.
-
+ (Array) tokenize(content, ptype = parser_type)
Tokenizes but does not parse the block of code.
Constructor Details
- (SourceParser) initialize(parser_type = SourceParser.parser_type, globals = nil)
Returns a new instance of SourceParser
403 404 405 406 407 408 |
# File 'lib/yard/parser/source_parser.rb', line 403 def initialize(parser_type = SourceParser.parser_type, globals1 = nil, globals2 = nil) globals = [true, false].include?(globals1) ? globals2 : globals1 @file = '(stdin)' @globals = globals || OpenStruct.new self.parser_type = parser_type end |
Class Attribute Details
+ (Symbol) parser_type
Returns the default parser type (defaults to :ruby)
79 80 81 |
# File 'lib/yard/parser/source_parser.rb', line 79 def parser_type @parser_type end |
+ (Object) parser_type_extensions=(value)
161 |
# File 'lib/yard/parser/source_parser.rb', line 161 def parser_type_extensions=(value) @@parser_type_extensions = value end |
+ (Object) parser_types=(value)
153 |
# File 'lib/yard/parser/source_parser.rb', line 153 def parser_types=(value) @@parser_types = value end |
Instance Attribute Details
- (String) contents (readonly)
Returns the contents of the file to be parsed
396 397 398 |
# File 'lib/yard/parser/source_parser.rb', line 396 def contents @contents end |
- (String) file
Returns the filename being parsed by the parser.
383 384 385 |
# File 'lib/yard/parser/source_parser.rb', line 383 def file @file end |
- (OpenStruct) globals (readonly)
Returns an open struct containing arbitrary global state shared between files and handlers.
392 393 394 |
# File 'lib/yard/parser/source_parser.rb', line 392 def globals @globals end |
- (Symbol) parser_type
Returns the parser type associated with the parser instance. This should be set by the constructor.
387 388 389 |
# File 'lib/yard/parser/source_parser.rb', line 387 def parser_type @parser_type end |
Class Method Details
+ (Proc) after_parse_file {|parser| ... }
Registers a callback to be called after an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called after the entire list of files is processed, see after_parse_list.
321 322 323 |
# File 'lib/yard/parser/source_parser.rb', line 321 def after_parse_file(&block) after_parse_file_callbacks << block end |
+ (Array<Proc>) after_parse_file_callbacks
Returns the list of callbacks to be called after parsing a file. Should only be used for testing.
349 350 351 |
# File 'lib/yard/parser/source_parser.rb', line 349 def after_parse_file_callbacks @after_parse_file_callbacks ||= [] end |
+ (Proc) after_parse_list {|files, globals| ... }
Registers a callback to be called after a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
255 256 257 |
# File 'lib/yard/parser/source_parser.rb', line 255 def after_parse_list(&block) after_parse_list_callbacks << block end |
+ (Array<Proc>) after_parse_list_callbacks
Returns the list of callbacks to be called after parsing a list of files. Should only be used for testing.
335 336 337 |
# File 'lib/yard/parser/source_parser.rb', line 335 def after_parse_list_callbacks @after_parse_list_callbacks ||= [] end |
+ (Proc) before_parse_file {|parser| ... }
Registers a callback to be called before an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called before the entire list of files is processed, see before_parse_list.
292 293 294 |
# File 'lib/yard/parser/source_parser.rb', line 292 def before_parse_file(&block) before_parse_file_callbacks << block end |
+ (Array<Proc>) before_parse_file_callbacks
Returns the list of callbacks to be called before parsing a file. Should only be used for testing.
342 343 344 |
# File 'lib/yard/parser/source_parser.rb', line 342 def before_parse_file_callbacks @before_parse_file_callbacks ||= [] end |
+ (Proc) before_parse_list {|files, globals| ... }
Registers a callback to be called before a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
231 232 233 |
# File 'lib/yard/parser/source_parser.rb', line 231 def before_parse_list(&block) before_parse_list_callbacks << block end |
+ (Array<Proc>) before_parse_list_callbacks
Returns the list of callbacks to be called before parsing a list of files. Should only be used for testing.
328 329 330 |
# File 'lib/yard/parser/source_parser.rb', line 328 def before_parse_list_callbacks @before_parse_list_callbacks ||= [] end |
+ (void) parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
This method returns an undefined value.
Parses a path or set of paths
93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 |
# File 'lib/yard/parser/source_parser.rb', line 93 def parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser") excluded = excluded.map do |path| case path when Regexp; path else Regexp.new(path.to_s, Regexp::IGNORECASE) end end files = [paths].flatten. map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }. map {|p| p.include?("*") ? Dir[p].sort_by {|f| f.length } : p }.flatten. reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } } log.enter_level(level) do parse_in_order(*files.uniq) end end |
+ (Object) parse_string(content, ptype = parser_type)
Parses a string content
116 117 118 |
# File 'lib/yard/parser/source_parser.rb', line 116 def parse_string(content, ptype = parser_type) new(ptype).parse(StringIO.new(content)) end |
+ (Symbol) parser_type_for_extension(extension)
Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.
168 169 170 171 172 173 |
# File 'lib/yard/parser/source_parser.rb', line 168 def parser_type_for_extension(extension) type = parser_type_extensions.find do |t, exts| [exts].flatten.any? {|ext| ext === extension } end validated_parser_type(type ? type.first : :ruby) end |
+ (void) register_parser_type(type, parser_klass, extensions = nil)
This method returns an undefined value.
Registers a new parser type.
139 140 141 142 143 144 145 |
# File 'lib/yard/parser/source_parser.rb', line 139 def register_parser_type(type, parser_klass, extensions = nil) unless Base > parser_klass raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base" end parser_type_extensions[type.to_sym] = extensions if extensions parser_types[type.to_sym] = parser_klass end |
+ (Array) tokenize(content, ptype = parser_type)
Tokenizes but does not parse the block of code
125 126 127 |
# File 'lib/yard/parser/source_parser.rb', line 125 def tokenize(content, ptype = parser_type) new(ptype).tokenize(content) end |
Instance Method Details
- (Object?) parse(content = __FILE__)
The main parser method. This should not be called directly. Instead, use the class methods parse and parse_string.
415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 |
# File 'lib/yard/parser/source_parser.rb', line 415 def parse(content = __FILE__) case content when String @file = File.cleanpath(content) content = convert_encoding(File.read_binary(file)) checksum = Registry.checksum_for(content) return if Registry.checksums[file] == checksum if Registry.checksums.has_key?(file) log.info "File '#{file}' was modified, re-processing..." end Registry.checksums[@file] = checksum self.parser_type = parser_type_for_filename(file) else content = content.read if content.respond_to? :read end @contents = content @parser = parser_class.new(content, file) self.class.before_parse_file_callbacks.each do |cb| return @parser if cb.call(self) == false end @parser.parse post_process self.class.after_parse_file_callbacks.each do |cb| cb.call(self) end @parser rescue ArgumentError, NotImplementedError => e log.warn("Cannot parse `#{file}': #{e.}") log.backtrace(e, :warn) rescue ParserSyntaxError => e log.warn(e..capitalize) log.backtrace(e, :warn) end |
- (Array) tokenize(content)
Tokenizes but does not parse the block of code using the current #parser_type
459 460 461 462 |
# File 'lib/yard/parser/source_parser.rb', line 459 def tokenize(content) @parser = parser_class.new(content, file) @parser.tokenize end |