• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Need help in programming....big time

P

PHPForever

n00b
Joined
Jan 11, 2005
Messages
22
I have been programming now for 5 years. I started using Turbo Pascal 6 & 7, later learned PHP 4 and then Java. I have abandoned TP and Java and have concentrated on PHP for the past 2-3 years. However now I’m interested in doing some really complicated stuff (or it seems complicated to me) and I need the guru’s advice and guidance.

Here is what I would like to learn to do and know in detail:

Bios services
I/O services (Reading and writing to the disk)
Write software to boot up an OS (I think it is called bootstrap)
Write a simple assembler.
Write a simple OS from scratch.

I have read over the following books and many others which I can’t even remember:

The Art of Assembly Language by Randall Hyde

Assembly Language Step-by-Step (The first and second edition)

Windows Assembly Language & System Programming (2nd edition) by Barry Kauler

Introduction to 80x86 Assembly Language and Computer Architecture by Richard C. Detmer.

What I have found is that each of those books introduces high level programming in low level. That is it! For example you would learn that there is no such a thing as an integer or a float, just zeros and ones (types are basically “the complier’s thing”) and an agreed upon way to manipulate them (a protocol or a standard). You would also learn something about data alignment, control structures, opcodes, function calls, memory models, etc…. a lot of things taken for granted in a high level language. That is absolutely great! I have learned how to write really fast assembly code. However, that is not what I’m after.

What I want to know is how exactly does say XASM’s IO functions work? How does DOS query those impossibly to query registers? How does DOS change the Instruction Counter? Where are those OS specific registers? How are they used?...stuff that books don't seem to explain, avoid to mention in the first place or say (in so many words) it is magic. – I don’t believe in magic :mad:

So guru’s I need your help. How did you learn to do that stuff? What books/resources did you read and recommend? What are the necessary beginning steps? What is the best way to learn this stuff? Any pitfalls to avoid?

Side note: I’m currently learning C so that I could try to understand the Linux C library. However, I’m afraid even after I learn C, I won’t fully understand the Linux C library.
 
Get an instruction set reference from your processor manufacturer of choice and read it, it should clear up a few points for you.
 
you do understand you won't be able to use php anymore then don't you? :cool:

Anyways, this is my recommendation. Learn the basics of assembly and the x86 architecture, but don't plan on actually writing your mini OS and shit in it. Theres only a few bits here and there that you have to do in assembly, but the rest you can do in C. If you are careful what you are doing, you can get almost a 1:1 representation of the assembly in your c code, with the major benefit that you won't have to slit your wrists pouring over shitloads of assembly.

Give up the windows ASM. It's a lost art and totally worthless.

Lastly, keep those little questions in the back of your mind, but don't really actively pursue them, or similar questions. Otherwise, you will want to hang yourself sooner or later.
 
Nevermind: That is to follow soon. Thz for your input.

Whatsisname: (1) As my user name suggests... PHP Forever. So no worry there...Perhaps I won't be working with PHP but working on PHP...thz.

Whatsisname: (2)
I agree with the rest of your post. All those assembly books don't make things easy, but what C books do you recommend to learn the stuff I'm interested in? I have read few books now and let me tell you, there wasn't anything I didn't already know! More over, the books that I have read repeat the samething over and over again...and considering I know assembly it is becomming really boaring and annoying....:mad: ... Not making any progess.
 
As I'm sure you know, you've got a long way to go. You've got the meaning behind a quite a few terms jumbled, but you'll get there in time. If you're really interested in operating systems I suggest that you do the following.

1) Learn C
2) Learn simple x86 assembly and be able to use it with C
3) Do all of the above using the gcc and it's associated tools.
4) Understand makefiles
5) Get yourself a copy of BOCHS. It's a great way to simulate a computer with out having to trash your own.
6) Read up on real mode operating systems. Look at other source (There are dozens of VERY simple hobby OSes out there. Consider looking at Linux 0.1, GeekOS (from my undergrad).
7) Using BOCHS, make a very simple flat file harddrive image and create a boot sector for it. Perhaps have it display "Hello, World!" on boot.
8) Write a BASIC BASIC real mode OS (single task, basic io)
9) Look into protected mode (this is HARD)
10) Try to write protected mode OS (VERY VERY hard).
 
I hear that Intel is dumping the Assembler BIOS spec and moving to a C spec soon. They claim that the current BIOS spec is defunct and will be replaced. The also said that because you need Assembler to write the BIOS, you dont see alot of progress (difficult to code GUI)

I would say use C to write all your stuff. Forget about BIOS for now, what are you gonna do with it? Sure you can write a nice interpreter in C ( most college upper-class labs usually have one or two of these labs... I know, I had to write a RPN calculator & a basic interpreted language engine in college )
 
if you want to write an OS, you WILL have to write some assembly at some point. There is no easy way to do context switching and setting ring levels w/o it.

I suggest that you do LOTS AND LOTS of reading. I feel like there are still terms that are easily getting confused here.

I imagine that there will be quite a bit of confusion if the bios started to offer functionality through a C-style callback api. While it's a step in the right direction, it will be a VERY slow step.
 
something i've wanted to try to do for awhile is to write a sort of emulator. Not a run time emulator, but something that does preprocessing and dissassembles the target, converts it to assembly for the machine that it will be run on, and reassemble. I bet doing that would make the victim program run quite a bit faster, and could be useful for emulating stuff like Mac programs, possibly build a version of OSX that would run on x86 and fast, because it would be running almost nativly.....

but its a lot of work and i don't have time for that :( :mad:
 
Ultimately, nothing that an OS does is "magic", at the end of it all, there are a series of in, and out opcodes and loads and writes to special memory locations. Part of the problem is getting documentation on which io ports, values, and memory addresses correspond to each device. The various jmp opcodes will overwrite the current instruction counter, and Intel has documented the steps for working with protected mode in their programming manuals.

Of course, your bios already wraps up a bunch of that stuff, and makes it available by apparently-magic register loads & interrupt calls, but ultimately your bios is just another program (beyond initial POST and starting the bootloader).

I doubt you want to write a bootloader, since GRUB and its ilk are readily available. If nothing else, I'd start with using GRUB (or another bootloader) and replace it later if I felt that it was necessary.

If you want to understand operating system development, I'd start by learning C and reading the driver & kernel source for an existing operating system. Many are available under an open-source license, and a small, 1-man project should be comparatively simple to understand. By reading the source to an existing OS, you'll learn how somebody else has tackled similar problems, and you'll learn some of the "magic" values to invoke certain devices.

This information is also described in documents like this and this. The first link describes Pentium-family processor operation, Volume 3 covers system programming considerations, like how to enter protected mode and configure page tables. The second link describes the ATA controller on the intel i925 chipset, you would need some understanding of how to manipulate PCI registers to make use of it, and would need to locate similar documentation for each piece of hardware you wanted to support (note: this documentation may not be publicly available).
 
ZeroX: Thz for your input. I didn’t know about BOCHS and I will certainly check it out. The other suggestions are really good. I was looking for small open source OSes and I found few links but I have to get through C first. I have a question for you – what C books have you read and/or recommend that are worth taking the time to read? Any C books touching on the subject of OS programming?

Cobalt2112: C is the way to go… I c now…just a little joke. I know that C is the choice of serious programmers. Question for you – Do you know any C books that deal with context switching?

MonkeyShave: I know about GRUB. I want to know at minimum the theory behind bootloaders, if any. Do you know of any books on bootloaders? I would really like to write one, even a crappy one, just for the experience. I will soon examine GRUB in details, but I rather do that after I have written a crappy bootloader simply because I want to make the mistakes and learn from them and compare what I have learned to what GRUB has done. Your post has been helpful…thz.

I would like to hear from all the gurus and make this thread as complete as possible. So plz don't stop here. :)
 
I haven't really used books to learn C. Between high school and college classes I learned the language. Books are hard to learn from compared to a class. If you can, try to take a class if you really want to learn.

As far as bootloaders go, for most (all?) x86 machines there is a defined procedure for bootloading. Here is a general overview (I think I have an example at home but I don't have access to it now)

1) BIOS checks through devices (in boot order)
2) BIOS checks the first sector for a boot sector (it has a special tag)
3) BIOS loads this into a know location in memory and jumps to it (BIOS is done it's part)
4) Execution begins with this first sector.
5) Typically the code will then pull in additional code/data. It may even switch to protected mode or harvest some real mode information for the OS it's about to load.
6) A kernel or some form is then loaded into memory (typically a high location in memory) and the bootloader jumps into this code.
7) Now the fun begins!


Here is a link to a chunk of my bookmarks. Lots of relevant stuff (as well as not so relevant stuff ;)) Save the file locally as I'll probably kill it within the week.
http://systemx.rauros.net/hardbookmarks.html
 
This book is one of the favorite C reference books. You can find it on ebay pretty cheap and it's definitely worth the money. Almost every student in my programming classes had a copy they took with them to the lab.

It doesn't go into detail explaining how to program but rather just how C does things. It's a great reference for getting the syntax right.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
I am not an expert in low-level C stuff, but I've been programming for a few years now, and the best way to learn something, IMO, is by mimicing. Just like we did when we learned to talk and walk and stuff like that. Grab as many sources off sourceforge of projects that you find interesting, or do something you want to do. Hit Grub's website and grab their source, just about anything for linux has it's source available, and Linux is a pretty full-featured OS, so it's going to have alot of projects that you'll find interesting. Learning about gcc is also a good idea, since it'll tell you about how your code gets translated to machine-code. Get ready to be frustrated and tired alot. Eat alot of carrots so you don't go blind, and stick with it. It's easy to pick syntax, but it'll take along time to get a working understanding of what you're doing. I hope this gives you some more ideas and helps you on your way. I wish I had a few years to learn asm, and low-level stuff, but such is life. For now, at least I have shaders and decent APIs.
 
There's nothing in particular about C that makes it suitable for performing a task/context switch, fundamentally, the operation is similar to pulling a rug out from under your own feet.

The specifics will depend on the cpu and the structure of your operating system, but in essence, it's a tricky data swap. You'll need a third piece of code (generally running as an interrupt handler or in supervisor mode) to actually perform the switch, and often you'll need to use (or abuse) some cpu features as well.

First, what is a context? A context is a combination of Program Counter (Instruction Counter, Instruction Pointer), General-Purpose Register Values, Program Stack, Processor Flags, and Address Space (Segments and/or Page Table).

In order to switch back to a task, you'll need some way to read this data out. Fortunately, on x86, the INT (interrupt) instruction will do much of that (read the opcode description in intel's docs). INT will write those onto the interrupt-handler's runtime stack, but you should be able to figure out how to manipulate that. Note that any other interrupt will also preserve this data (not just software interrupts called from your code), so you could trigger a task switch from a timer or keyboard interrupt as well.

Of course, you'll want to store this context data someplace special in memory (not as part of the context's memory allocation), many operating systems reserve a global table (in kernel-space) for context records.

In order to switch to a task, you could use the IRET opcode. Note that it does the inverse of what INT does, namely, it reads values from its runtime stack and writes them into your flags, Program Counter, and Segment registers (the details depend on some cpu mode bits). So to switch into a task, you'll need to prepare your stack with the proper values (taken from previously-stored data) and invoke IRET.

386 and later CPUs also support hardware task switching, via the Task Switch Structure, but this remains unpopular.

Now that I've done a little web search, this page describes the process in a little more detail. You may also want to look at the newsgroup alt.os.development, it wasn't very active (when I last read it) but the archives could answer many of your questions.
 
MonkeyShave: Thanks a lot. News groups were completly out of my mind. Also thz for the context switching link.

I now kind of get the big picture. It seems to me that the knowledge of the CPU's features is more or less priceless. Everything else depends on the developer's imagination, best practice, and few little tricks.

Thz for all your help gurus, programmers...people. I will try to post any additional resources and books worth posting.

Out.
 
You must log in or register to reply here.
Back
Top