About autotools, Linux CD XA driver and ... me :)

  • Thread starter Thread starter einherjar
  • Start date Start date
Status
Not open for further replies.
E

einherjar

Guest
Hello there !

As you may have heard, I am planning to spend some time on Q-Gears development.
I'm french, studied computer science and am now working in an international
CAD/CAM/PLM company. I have a strong knowledge of C/C++, OpenGL and SDL;
my prefered platform of development is linux. As many (all?) of you, I am
a confirmed FF7 fan.

I already commited some modifications to the SVN repository, mainly:

1) an autotool infrastructure, which relies on autoconf, automake and libtool;
I could describe you how to use it, but I prefer to redirect you to the guidelines
I wrote in the files named README, INSTALL and HACKING. If you have any
problem, or if you want a quickstart guide to autotools, do not hesitate to tell me.

2) a FileDriver implementation that allows Q-Gears to read original Playstation
CDs, which use an iso9660 filesystem recorded in XA mode. I first tried to use
libcdio, but it still does not support such CDs very well -- at least, not the CDs
of the FF7 PAL version I own.

... So, I am pleased to join you, and I will do my best to help you make Q-Gears succeed :-D
 
We believed you but it's nice to see him post a small blurb.

the auto tools though are most helpful too moi :D

Cyb
 
Meh, the "include everything from src root" thing broke my vs2005 project files.
vs2005 compiler sets the current directory to the source directory, so they don't work there. I will try to find a way to add "project root" as an additional include dir, but not sure if that will be possible (in that case it would either need #ifs or a branch from the project trunk).

EDIT: Ok it was easy. :p
 
Last edited:
I was going to say....

Branching from trunk is a very bad move. Forking is a last-resort move that must be avoided at all costs. If there is a design that simply is unworkable, take the code and create a fork in another reposotory.

If you create a fork off the main tree, patches will have to be constently pulled from one to the other. This violates the DRY princaple (Do not repeat yourself)

The only time branching should be considered is of we reached 1.0 and wholly new functionality is to be added. I have stated that the best course of action is to get Q-gears running as close to the original executable as possable. Things like extended graphics, mp3 support, and other fancy stuff should be held off. When the refrence model is finished, then fork and go wild with new functionality. This means we can keep 1.0 around and offer simple bugfixes (That can be rolled into the new branch). The extra stuff can be experiments that can melt down without effecting the current codebase.

Does that make sense tp everyone?
 
I always follow the KIWS principle.
Keep It Working Silly.

This allows complete rewrites of the new branch whilst the old one seems to work.

Anyhow this is a bit down the road in any case.  Sigh need to update my code again. :P

Cyb :D
 
1) Why we need use full path to files? I can't find that option in Dev C++ and I cant upload any of my changes because of that.
2) I'm strongly against makefiles everywhere. We need to cleanup things.
 
> 1) Why we need use full path to files?

Relative paths are problematic (== it may not compile)  when using a build directory different
than the source directory in an autotools context.

> I can't find that option in Dev C++

You just have to add the root directory of q-gears to the project's include path (as a `-I' option to g++).
In DevC++ 4.9.9.2, click on the "Project / Project Option" sub-menu;
in the "Project Option" pannel, select the "Directories" tab, then the "Include Directories" sub-tab;
add the root source directory (something like:
C:\Documents and Settings\einherjar\My Documents\q-gears\trunk\src
C:\Documents and Settings\einherjar\My Documents\q-gears\trunk)

> I cant upload any of my changes because of that.

Argh!
I sincerely hope the process above will fix your problem.

> 2) I'm strongly against makefiles everywhere. We need to cleanup things.

Hmmmm. If you are talking about the Makefile.am files, it's gonna be problematic:
automake needs to have one in each subdirectory.
 
Last edited:
> 1) Why we need use full path to files?

Relative paths are problematic (== it may not compile)  when using a build directory different
than the source directory in an autotools context.

> 2) I'm strongly against makefiles everywhere. We need to cleanup things.

Hmmmm. If you are talking about the Makefile.am files, it's gonna be problematic:
automake needs to have one in each subdirectory.
Click to expand...
Hmm I think the trouble is automake. As I understand it used to create makefile/compile project. I think there a two ways out of this:
1) Maintainer for each version who create makefile and update them according to last changes (he could use automake or something)
2) Use some enviroment wich has same project files under linux and windows. code::blocks maybe?

> I can't find that option in Dev C++

You just have to add the root directory of q-gears to the project's include path (as a `-I' option to g++).
In DevC++ 4.9.9.2, click on the "Project / Project Option" sub-menu;
in the "Project Option" pannel, select the "Directories" tab, then the "Include Directories" sub-tab;
add the root source directory (something like:
C:\Documents and Settings\einherjar\My Documents\q-gears\trunk\src
C:\Documents and Settings\einherjar\My Documents\q-gears\trunk)

> I cant upload any of my changes because of that.

Argh!
I sincerely hope the process above will fix your problem.
Click to expand...
I found this option but there are problems with it. It setted from disk so project must be lokated in the same directory for all users (or they need to change this)... this is very bad. And it won't work if directory in path has spaces (at least for me). So it's very baaad =(
Code::BLOCKS may be good solution.
 
Is it really that needed to do that with the includes? I mean, it breaks relativity (you can't rename a dir, or move a class arround without changing all the #includes), and makes it harder for everyone xcept for automake. I think it would have been nicer to just try to get a working make/build script or something like that...
 
> Akari:
> It setted from disk so project must be lokated in the same directory
> for all user

Yes, I agree. I was a little reluctant to suggest it, as it did not satisfied me either. I searched a little, and simply adding "."  to
include directories works for me -- could you please try this?

(Actually, it will not compile due to the directive #include "config.h" in ffvii/filesystem/GameFileSystem.h; don't hesitate to
comment it out and commit this modification, we will find a way to work this around later)

> gigaherz:
> Is it really that needed to do that with the includes? I mean, it breaks relativity

If it really troubles you, I can search for a way to have relativity work in automake, but it does not like it very much --
and it is not the "usual" way either.

> I think it would have been nicer to just try to get a working make/build script or something like that...

(The idea is precisely to let automake and autoconf build these scripts)

> Akari:
> Hmm I think the trouble is automake.

Under linux (or unix), autoconf/automake is the most conventional way to setup a project; among many others,
gcc, gnu bison, The Gimp, vlc use it. The pair can automatically create ready-to-use tarballs, thanks to which
a user (not a developper) only has to type `./configure' then `make' to have its system checked for dependencies
and the project compiled. Removing automake means removing autotools, and I personaly think that this would be
quite sad :-D

Anyway, what's your point against Makefile.am? I can handle their maintenance, and I can also write some guidelines
to describe how to update them. We could "hide them" from you in a dedicated directory, and let the `bootstrap'
script move them to their correct places -- I am not very fan of the idea, but this could be an intermediate trick
to please everybody :-D
 
Watching from the sidelines, it would seem we have run across a speed bump. It's good we ran into this problem sooner rather than later. However, there is a reason for debate, so I'm throwing my hat into the ring.

We have run into another impasse where two programming philosophies are at ends with each other. Getting down to the brass tacks,  the problem is project management, which brings me into the picture.

I'm going to lay down the problem proper, and then offer a resolution. From this I would like comments on how to best implement the solution, so that everyone is happy.

Now I fancy myself as the ground layer. I come up with ideas on how implement Q-gears to the best of my ability. At the core, I'm a human resources guy, and don't like making decrees on high on what should and shouldn't be implemented without a lot of explanation on my part. I also am humbled by the fact that there are people who are willing to work on my silly idea and surprised by the support.

That being said.

Autoconf/Autotools stays, in all it's wretched, directory littering ugliness.

Sorry, that's how it is, any attempt to remove it will result in an immediate reversion on SVN to put it back.

Like I said, I'm not one to make decrees on high, but this issue is really non-negotiable. Now as opposed to just saying that this is how it is, and letting it be, I'm going to very carefully explain myself, and make the case for everyone here to understand.

Autotools, as it name implies, is a set of automatic tools to make a programmer's life a whole hell of a lot easier. It's also ugly as sin, but it's really a necessary evil. It's very critical part of cross-platfrom programming, and is a standard in software pre-configuration prior to compile-time. At the heart of Autotools is a macro language, called M4, which when used in conjunction with scripting, allows source code to become "aware" of it's surrounding environment. This allows the source code to *rewrite itself* in a way based on the computer it's been unpacked in.

For example, when you have autoconf set up in your application, a ./configire script will run a diagnostic on your computer system. It will check the Operating Syetem, CPU type, graphics abilities, installed libraries, and what ever else you think is critical to the application. From this system check, makefiles are generated *automatically* based on what kind of system it's on. There's no need to hand tweak library names, or to have to play "catch up" with the makefiles when something in the system's changed.

The issue at heart is this; Autotools is a very non-IDE way of doing cross-platform project management. It's ugly and harder than sin to work with if you don't understand what it's doing. The problem is that the system works for almost every platform. As opposed to fighting it, and calling it a blight on your code, you need to be *using* it. If it doesn't like spaces in your directory names, well then you need to change your directory names. If you don't like that it requires a file named "configure.h" when we already have a file named that which does something else, we need to rename our file to something else so autoconf can be used correctly.

Most every mature Linux application uses autotools, from glibc, to X, to gcc to the Linux kernel itself. Keep in mind that glibc and gcc are probably the most ported application in the entire Open Source ecosystem. That means something is being done correctly.

I do not want to have an IDE as a developmental dependency. (To program Q-Gears, you need Code::BLOCKS). That kind of thinking is ludicrous in the *NIX world. Using DevC++ is causing enough heartaches enough already. (Nothing against you choice in IDEs Akari, I'm sure it's a nice system). However, when I was a programmer once upon a time, my boss and I used nothing but "vi" to manage the codebase of  an entire enterprise-level database system.

Now it's late, and I think I'm ranting. (If I am, I'm sorry, I will probably look at this tomorrow and pound my head on the table for writing all of this.)

Anyway, I open the thread on how best cooperate with our new found makefile making friend. I'm also open to criticism too. Keep in mind, autoconf can make windows makefiles too!
 
My 200 cents worth here.

1) Automake is a definate must.  Especially for ease of portability concerns.  It is difficult to initially deal with  and I have yet to tame this monster myself.

2) Although I love IDE's they do one very big diservice, they make a lazy programers life easier.   It's not personal mind you I've been bitten by the IDE bug, so I know what I am really saying.  It covers up many SINS especially from the perspective of setting things up.  "The IDE handles everything" is actually the root of the problem here.

An example is porting an application written in the Turbo C++ environment for an X86 and a PC104 board to ARM thumb AT91SAM7X128.
We have different compilors with different behaviors and completely different platforms as well as completely different IO spaces.
Physically it is a bit of twisted wiring on a PC board.
Programatically all the low level IO needs to be rewritten.
I also couldn't use the old make file period.

So I had to use Eclipse and move all the code into it's own proprietary format, etc. etc.  It's a bit of a nightmare.  I also had to remove all the custom code unique to Turbo C++ (IE port usage) to use MACRO's that maped into the address space of the ARM.  Then I had to play with all the objects (because C++ and in place execution of code can be quite insane).  Object instantiation order is a BIG problem in embeded applications (if the object that uses another object is created before the object it uses is, we have a potentially deadly and permanent program failure).

This took 2 weeks.  I hope you can see my point here.  The fact is I KNOW what I am doing and it took me 2 weeks to port.  Now lets talk about someone utterly clueless, all right enough said isn't that?  They'll just either give up in frustration or bug the heck out of people who "have better things to do".

The truth is spending time to do this part will make things significantly less anoying later on for everyone (Akari inclusive).  No it won't be easy, not it's not exciting, no it won't give you kudos or praises from anyone.  However it will get you beyond the current state of things, for example my new code doesn't work because, I can't get it to compile on my platform.  Likely this will be fixed eventually (GL and SDL seem to be the bulk of the pain), like everyone I'm a bit lazy, and doing a lot of twiddling is not on my priority list unfortunately.   Reinstalling Linux just to fix it seems kind of like shooting myself in the foot because my toe itchs.  So there you have it. :)

This is the sort of thing such as getting a drivers license to operate an automobile one needs to do in order to move on with things in a neat and ordeerly manner.   Some pain now will be significantly less suffering later is the best way to look at it.

Cyb
 
I think I exaplined myself way too poorly before...

I never wanted anyoen to remove automake, it's a really useful tool, and when so many projects use it, that means it works.
What I don't really like is that way of doing the includes. Mostly because it's the FIRST time I see it, even though I have read or used code from a few projects that did use automake. Maybe I was kind of lucky (or un-lucky) that none of them did, but I have always coded using relative paths for includes so that's what I'm used to do now, can't do much about it xcept getting used to it for this project (I personally will keep using relative includes in other coding, at least for now).
Also in Visual Studio it's easy to setup, it has an environment variable for the project directory, so that's not a problem to work with QGears.
 
Yea, you will have to excuse our mess, we are still kind of figuring out programming disciplines here. :P
 
I was just making a statement as to why automake, no one in particular was cited (save Akari :) ).
I do need to have more discipline in what I have been doing.  Documentation is a must.
I have looked over my code, it might need to use a bit clearer documentation.

IDE wise there really is no ideal one, Eclipse has built in CVS support, but CVS is not only old it can be extremely bad news, and with a single mistake our entire source tree could be utter toast.

The Eclipse community appears to be having problems and seems to be willy nilly all over in there developement.  I can LEARN from other's mistakes.  Too me what's going on with Eclipse is a dilution of there direction.  The same can happen with Q-gears.  A working engine is important but getting there needs to be carefully ploted.  An Axiom from engineering and prototyping things is measure twice cut once.  It means plan things out in advance.  I appreciate the hard work and quick progress we have had, my problem is I'm somewhat lost at the moment.

It's Ok to be lost a little bit I suppose, however I have to play catchup, and that can be very difficult for me.  That being said I have (unfortunately) more time so I'll be squishing around with the code as I can.

Cyb
 
What I don't really like is that way of doing the includes. Mostly because it's the FIRST time I see it,
even though I have read or used code from a few projects that did use automake.
Click to expand...
I was not used to it either. Actually, it is an easy way to make the following work:

  $ tar xvzf foobar-0.1.tar.gz
  $ cd foobar-0.1
  $ mkdir build
  $ cd build
  $ ../configure
  $ make

That is, having a build tree different from the source tree; it is
interesting in several situations, most notably when wanting several builds --
with different compiling options -- for the same sources' location.

Also in Visual Studio it's easy to setup, it has an environment variable for the project directory, so that's not a problem to work with QGears.
Click to expand...
Yep, and for DevC++, adding "." to the include path should work
 
Last edited:
I looked through some projects that use automake and none of them littering their sources. There only one automake in root that looks something like that:

Code: [Select]
Code: 
AUTOMAKE_OPTIONS = subdir-objectsbin_PROGRAMS = tmwtmw_SOURCES = gui/widgets/dropdown.cpp \       gui/widgets/dropdown.h \       gui/browserbox.cpp \       gui/browserbox.h \       gui/buddywindow.cpp \       gui/buddywindow.h \       gui/button.cpp \       gui/button.h \       gui/buy.cpp \       gui/buy.h \       gui/buysell.cpp \
And they use normal way to include files...
 
littering their sources.
Click to expand...
Doh. Fair enough, I suppose :|

OK, let me make a proposal:
1) I revert my changes on #include to match your (<- plural) prefered style,
  after having modified what needs to be so that buildir != srcdir still works;
2) I use the subdir-objects option to limit the number of Makefile.am;
  still, the followings will remain:

q-gears/Makefile.am

q-gears/utils/Makefile.am
q-gears/utils/scriptdump/Makefile.am

q-gears/src/Makefile.am
q-gears/src/common/Makefile.am
q-gears/src/ffvii/Makefile.am
q-gears/src/xeno/Makefile.am

q-gears/output/Makefile.am
q-gears/output/ffvii/Makefile.am
q-gears/output/xeno/Makefile.am

What do you think about this? I imagine that (1) is OK for everybody; but what about (2)?
 
Last edited:
littering their sources.
Click to expand...
Doh. Fair enough, I suppose :|

OK, let me make a proposal:
1) I revert my changes on #include to match your (<- plural) prefered style,
  after having modified what needs to be so that buildir != srcdir still works;
2) I use the subdir-objects option to limit the number of Makefile.am;
  still, the followings will remain:

q-gears/Makefile.am

q-gears/utils/Makefile.am
q-gears/utils/scriptdump/Makefile.am

q-gears/src/Makefile.am
q-gears/src/common/Makefile.am
q-gears/src/ffvii/Makefile.am
q-gears/src/xeno/Makefile.am

q-gears/output/Makefile.am
q-gears/output/ffvii/Makefile.am
q-gears/output/xeno/Makefile.am

What do you think about this? I imagine that (1) is OK for everybody; but what about (2)?
Click to expand...
I think it's a good compromise. Let's stick this way. I think this will please everyone =)
Can't wait to see this in SVN. I upload all my changes right after it.
 
Status
Not open for further replies.
Back
Top