Tuesday, December 8, 2009

Turbo parser example

Here is a example of counting tokens from a file with Turbo parser.
Create a new VCL forms application, drop a TOpenDialog(rename it to OpenDialog) and a TButton on the form, double-click Button1 and copy-paste this code:
procedure TForm1.Button1Click(Sender: TObject);
// token record that holds the type and count
type TTokenRecord = record
                  Token : TurboTokenType;
                  Count : Integer;
     end;

// array of tokens
type TTokenRecordArray = array of TTokenRecord;

var Parser : TurboParser;
    Token : TurboTokenType;
    TokenArr : TTokenRecordArray;

      // this procedure tries to find "Token" type in
      // TokenArr and increment the count.
      // if "Token" type does not exist in array then
      // it creates a entry for it by increasing the
      // number of elements in "TokenArr"
      procedure CountThisToken;
      var index : Integer;
      begin
           for index := Low(TokenArr) to High(TokenArr) do
               if TokenArr[index].Token = Token then begin
                  Inc(TokenArr[index].Count);
                  Exit;
               end;// if TokenArr[i].Token = Token then begin
           SetLength(TokenArr, Length(TokenArr) +1);
           TokenArr[High(TokenArr)].Token := Token;
           TokenArr[High(TokenArr)].Count := 1;
      end;// procedure CountThisToken;

      // this procedure loops through all elements
      // in "TokenArr" and creates a formated string
      // Token "NAME" Count = VALUE + LineBreak
      // Token "NAME" Count = VALUE ...
      procedure ShowCounters;
      var s : string;
          i : integer;
      begin
           s := EmptyStr;
           for i := Low(TokenArr) to High(TokenArr) do
               s := s + Format('Token "%s" Count = %d', [
                 TurboTokenToString(TokenArr[i].Token), TokenArr[i].Count]) + #$D#$A;
           ShowMessage(s);
      end;// procedure ShowCounters;

begin
     if OpenDialog.Execute then begin
        // create a instance of TurboParser class and
        // assign the filename
        Parser := TurboParser.Create(OpenDialog.FileName);
        // start a loop
        while True do begin
              // grab next token
              Token := Parser.GetNextToken;
              // if reached EOF(End-Of-File) or internal error
              // then exit loop
              if Token in [ttEndOfFile, ttInternalError] then
                 Break;
              // find and increment the count of this token
              CountThisToken;
        end;// while True do begin
        // show a formated message with all counters
        ShowCounters;
        // free allocated memory to parser
        FreeAndNil(Parser);
        // set the length of "TokenArr" to zero
        SetLength(TokenArr, 0);
     end;// if OpenDialog.Execute then begin
end;
This is a demo file which on which I've tested the above code(wrote in 7-8 minutes, yes it's that easy to use TurboParser -- which can be easily modified and/or extended)
application TurboParserExample
{
 include
 {
  "filename.extension",
  "otherfilename.extension"
 }

 function main
 {

 }
}
and here is the message I got
---------------------------
Project1
---------------------------
Token "application" Count = 1
Token "" Count = 6
Token "{" Count = 3
Token "include" Count = 1
Token "" Count = 5
Token "." Count = 2
Token "}" Count = 3
Token "function" Count = 1

---------------------------
OK   
---------------------------

2 comments:

  1. Has the Turbo Parser project been abandonned?

    ReplyDelete
    Replies
    1. In a nutshell, yes, I still have it on my radar... just need some free time...

      Delete

Blogroll(General programming and Delphi feeds)