[Mapserver-users] maplexer and mapparser

[Steve Lime]

Changes made to the C files would be lost, not the .l and .y files. The
.c files are generated when I create builds either nightly or official
releases. That's done so you won't need flex or bison. If you build from
CVS then you will need those tools.

We also have a desire to generalize reading map files from memory,
either as a whole or in part. There is already a bug filed and I've got
some code written, but no time frame yet. Other things just keep coming
up. You shouldn't need to touch maplexer.c, just code in mapfile.c. The
lexer can work from a string buffer, and in fact that's how parsing
expressions or URL-based configuration works. Check out the
msLoadMapString function in mapfile.c for an example of working from a
string and not a file.

It's a good idea, you're just ahead of the game at the moment.

Steve


>>> "Andrew Arace" <AArace at geonetics.com> 12/19/2003 1:38:06 PM >>>
I appreciate the explanation, and yes, it helps thanks.
You say "Any changes made [in maplexer.l and mapparser.y] would be
lost
anyway" but how would they be lost?
I'm using my own build environment on windows, so I'm not running flex
or bison on those files at any time. 
I'm currently not changing them at all, just trying to understand how
they work.
 
Mainly, what I've been implementing, is the ability to read map files
from a url instead of a static file on the hdd.
This will let me send the map= parameter as
map=http://server/getmap.asp  
to be able to geneate a text/plain response which is the map file. In
this way, I can utilize asp's (or any web
language's) database connections, session, etc to write back the map
file syntax. hence dynamic on-the-fly maps.
 
The functionality for reading data from http was already built in
(using cURL) and I utilized the function calls
that it provided.
Right now, I have it functioning, but in order "msyylex" the new
'virtual' map file, it had to be FILE on 
the disk. So currently, when fetching the map file response from the
URL, it is saving it to temp directory.
 
My goal was to keep the map file in memory when it is read from the
url
response, but I could not understand
the maplexer.c enough to change it (because I see that it is only
written to  read from files)
 
What do you think of my approach, and how I should proceed for the
virtual map file?
 
 
--
Andrew Arace
aarace at geonetics.com 
Software Engineer
Geonetics, Inc
(617) 896 - 4409

>>> "Steve Lime" <steve.lime at dnr.state.mn.us> 2:22:11 PM Friday,
December 19, 2003 >>>

Hi Andrew. The files maplexer.c and mapparser.c are generated
automatically by the tools flex and bison respectively, and in turn
are
based on the configuration files maplexer.l and mapparser.y. I've
never
looked at the C source for those files, nor would I want to. Any
changes
made there would be lost anyway. Here's what each file is used for:

  - maplexer.l contains regular expressions and C snippets to break a
map file, certain CGI parameters and MapServer expressions into
tokens.
These tokens are then handed off to other sections of the code to be
used. The lexer keeps track of where it is in a file or string and the
function msyylex simply gets the next token. The C code generated by
flex uses lots of global variables (and is not thread safe) to hold
various pieces of information about a token such as it's string
representation.

  - mapparser.y defines the grammar, the rules, for MapServer logical
expressions. It has nothing to do with parsing map files, only
expressions. It's cryptic to look at but once you figure it out it's
not
bad. The grammar consists of things like setting operator precedance
(i.e evaluate * before +) and sets the rules for doing things like
adding 2 numbers, or comparing strings. The parser takes tokens from
the
lexer and input. There's a function called msyyparse that evaluates
any
particular string against the grammar.

Does this help?

Steve 

>>> "Andrew Arace" <AArace at geonetics.com> 12/18/2003 4:59:28 PM >>>
I'm currently working on some modifications to the MapServer source
code, and I've built the whole project in windows 2000, using the .NET
IDE.
I've got the exe, the necessary functions run great.

However, my current modifications require me to understand the
maplexer.c and mapparser.c.
I'm a pretty smart guy. But I'm a Unix noobie. (I know, some will
think
that's a contradiction...)

Anyway, I have NO IDEA what is going on in those files. No clue.
People
have told me what yacc is, and bison, but I don't have a very good
understanding, and these files are the root of all my confusion.

Basically, what are these used for. what is actually going on when the
mapfile parser calls switch(msyylex()) {...} (mapfile.c,
loadMapInternal())

What are the crazy int arrays in the maplexer? such as yy_accept[2174]

?  What are the maplexer.l and mapparser.y files?

I'm just in need of a thorough explanation, please please? 

Thanks very much in advance

--
Andrew Arace
aarace at geonetics.com 
Software Engineer
Geonetics, Inc
(617) 896 - 4409

Disclaimer Notice: This message contains confidential information and
is intended only for the individual named. If you are not the named
addressee you should not disseminate, distribute, or copy this e-mail.
Please notify the sender immediately if you have received this e-mail
by
mistake and delete this e-mail from your system. E-mail transmission
cannot be guaranteed to be secure or error free as information can be
intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
contain viruses. The sender therefore does not accept liability for
any
errors or omissions in the contents of this message, which arise as a
result of e-mail transmission. If verification is required please
request a hard copy version. BSC Companies, Inc.; BSC Group, Inc.;
Geonetics, Inc. 15 Elkins Street Boston, Massachusetts 02127 USA.
www.BSCGroup.com. 
http://www.bscgroup.com/ 


Disclaimer Notice: This message contains confidential information and
is intended only for the individual named. If you are not the named
addressee you should not disseminate, distribute, or copy this e-mail.
Please notify the sender immediately if you have received this e-mail by
mistake and delete this e-mail from your system. E-mail transmission
cannot be guaranteed to be secure or error free as information can be
intercepted, corrupted, lost, destroyed, arrive late or incomplete, or
contain viruses. The sender therefore does not accept liability for any
errors or omissions in the contents of this message, which arise as a
result of e-mail transmission. If verification is required please
request a hard copy version. BSC Companies, Inc.; BSC Group, Inc.;
Geonetics, Inc. 15 Elkins Street Boston, Massachusetts 02127 USA.
www.BSCGroup.com. 
http://www.bscgroup.com/