Is there a way to convert Linux C++ lib files to code?

[Red Squirrel]

Oh geez, I think you need to go read some of the basics of compiling and linking. A header merely provides signatures of what the library contains (not 100% true, but true enough here). To steal from this stackoverflow answer, "The header is a phone number you can call, while the library is the actual person you can reach there!"

Extending that a bit to give you a super short crash course in what happens when you "build", there are two main phases to a build, compiling and linking. I'll skip preprocessing, how template metaprogramming is compiled, etc. The short version of compiling: taking various bits of source code, e.g. C/C++, and transforming them into some output format, e.g. binary machine code. The short version of linking, biased towards statically linking: taking the various output format pieces and smashing them together in such a way that references between the various pieces may be resolved within the self-contained binary file.

Extending the phone number/person you call analogy, at the end of compiling you have binary code that says "when I want to use so and so function I call this phone number with this extension to reach them". Linking is like moving all of you into the same building, then replacing that phone number with their extension alone so you can call them within your building. Since the header just lets you know that there's a phone number to reach someone, it doesn't do you any good to just have that if there isn't actually someone you can reach.

This is a very abbreviated view of the process and not 100% technically correct, but hopefully it helps you start understanding why you need a library (not even sure what you mean by that - pure c/c++ code? pure machine code? pure intermediate language code?). If you mean "why can't I just copy their c/c++ source code and use it directly in my program - you can, but the point of a library is to give you a self-contained unit that has certain known points you may contact and use, hiding you from what's going on inside that unit.

I'd strongly suggest reading up about the phases of building an executable before mucking about further. It will serve to help you significantly in any future build issues you encounter.

I get that part, but what I'm trying to understand is why can't the stuff in the library be performed in regular code?

Ex: Say I want to write a header/class that does XYZ, I can write it all out in code, and ship it as a single header file (or a folder with multiple header/cpp files if I want to organize it better) that has everything required in it for it to work. No "install" required, just include the main header in your program and that header will include what it needs. Typically I create a .h and .cpp but I'm not even talking about that right now, as technically you can just put it all in one if you really wanted to. I know the .h that comes with libs still need the libs, but why can't they impliment what the libs do, in the .h file or other file that is called by the h file so that it's stand alone?

What does a library do special, that can't be done by simply writing it in a single set of header files so that everything needed for XYZ to work is written in normal code?

Say I'm a dev that works for MySQL, so I want to make a connector so people can use it, what stops me from simply implementing the MySQL protocol entirely in C++ and simply ship the code directly as a set of header files instead of making a lib? Do libs make certain special system calls that can't be done in plain C/C++?

C++ tutorials never really talk about the lib stuff. College definitely didn't talk about any of that either.

 

[devman]

Squirrel, I can sympathize, it was a culture shock when I graduated and went to work as a software engineer. There is a world of difference between creating cli programs for assignments that compile with simple Makefiles and actually creating software packages that install on real world systems. The latter requires build engineering and tools that one is simply not exposed to in undergrad.

I will say that these build processes exist for a good reason and trying to work around them instead of embracing them will only harm you in the long run.

 

[Tawnos]

I get that part, but what I'm trying to understand is why can't the stuff in the library be performed in regular code?

You obviously don't get that part, as I explained what happens when you get a library and you breezed right by it. The library contains the object code used by that library and the symbol table that allows it all to be fixed up when you combine the library together with your program's compiled object code.

Ex: Say I want to write a header/class that does XYZ, I can write it all out in code, and ship it as a single header file (or a folder with multiple header/cpp files if I want to organize it better) that has everything required in it for it to work. No "install" required, just include the main header in your program and that header will include what it needs. Typically I create a .h and .cpp but I'm not even talking about that right now, as technically you can just put it all in one if you really wanted to. I know the .h that comes with libs still need the libs, but why can't they impliment what the libs do, in the .h file or other file that is called by the h file so that it's stand alone?

What does a library do special, that can't be done by simply writing it in a single set of header files so that everything needed for XYZ to work is written in normal code?

Say I'm a dev that works for MySQL, so I want to make a connector so people can use it, what stops me from simply implementing the MySQL protocol entirely in C++ and simply ship the code directly as a set of header files instead of making a lib? Do libs make certain special system calls that can't be done in plain C/C++?

C++ tutorials never really talk about the lib stuff. College definitely didn't talk about any of that either.

The header files (generally) don't include code, just definitions. When you do the build step, you are taking the source code and compiling it into an object file. You _could_ just use the source code you downloaded and add your files within the tree/to the build definition, but separating it allows you to compile the object code for that library once and simply link it with your program. It separates the piece you wrote from the piece that you're using. Imagine using multiple libraries if it meant having to include all of their source in the build - every time you get a library, you copy it into the source tree, then adjust your build scripts to merge the library's source files, and hope that everything still works when you're done.

Further, it's impractical to put all of the source in the headers - that would mean putting all the source from all of the files into a single file. Not to mention that when a file is being compiled you'd have your intermediate source file (part of the preprocessor stuff I didn't go into) become insanely larger as the entirety of a source tree used for that header is prepended to every file you compile when the compiler replaces the #include directive with that code. If you want to see what that looks like, try running gcc -E files > intermediateOutput.cpp

 

[jimmyb]

but why can't they impliment what the libs do, in the .h file or other file that is called by the h file so that it's stand alone?

Strictly speaking it could be possible to move the entire implementation in a .h/.c file that was #included.

There are many disadvantages to this approach. Let's assume your program uses a bunch of libraries (ie. mysql, stdio, libc, etc.):
1) Much harder to develop the library in the first place. All your source has to descend from a single .h file. Developers like to separate their programs into separate files and then use a linker to combine then.

2) It now takes you an hour to compile your program for every little change you make.

3) Each program has its own version of the libraries inside it. Your otherwise kB-sized program is now MB-sized. This also uses up a proportionally larger amount of memory when running.

4) If there is an update to a library it requires recompiling everything on the system that uses it. Imagine if there was a bug fix to C standard library - now you have to recompile everything on your system if you want that fix.


Advantages are:
1) You don't have to build libraries and use a linker.

 

[Red Squirrel]

The advantages outweigh the disadvantages though, so, is it actually possible?

Also I think maybe there was a terminology issue on my part but when I say header file I'm strictly speaking about a file that is included which has code in it. Ex:

#include "helloworld.h"

int main()
{

HelloWorld * Msg = new HelloWorld();
Msg->Display();
delete Msg;
return 0;
}

helloworld.h:

// (here I can possibly include required standard header for any other functions/classes my own class uses or leave it up to the programmer using my "library" to include it themselves)
class HelloWorld
{
public:
HelloWorld();
~HelloWorld();
void Display();
};

HelloWorld::HelloWorld()
{
//constructor stuff
}

HelloWorld::~HelloWorld()
{
}

HelloWorld::Display()
{
cout<<"Hello world";
}

Now from what I understand a library header file would only have the top portion and the actual "code/functions" is machine code in the lib file. But why is it that they can't implement it as just code and put it as part of the include file? Is it because it's actually not written in C++ but rather assembly or other language then converted to machine code?

A .h file is not by definition only declarations of something, it can be anything. So what I'm asking is what makes it so they can't implement stuff 100% in the form of code instead of having to use libraries. It would simplify everything for both parties. Now typically you'd separate the declarations in a .h and put the functions and actual work in .cpp or w/e but that's just details and it can work either way. Typically it's how I do it myself but it will still work if you put it all in one header.

I hope my question is more clear. I'm just trying to learn why they do that, and if by chance there is a way to take a tar.gz of a library and instead of installing it into the system, generate the required code that I can simply include right in my program, and idealy, ship. Or perhaps if it's just a bad idea, then why.

I'd also like to learn more general stuff about how this lib stuff works in general, I think I'm sorta starting to understand but a more in debt tutorial would be nice. Not only for my own programs but for Linux in general, there are often times where I simply can't get a program to install due to a dependency issue but knowing more how that stuff works would make it easier to troubleshoot those things.

 

[jimmyb]

so, is it actually possible?

Technically I suppose. The hurdles of doing this far exceed becoming proficient in the compiler toolchain.

A .h file is not by definition only declarations of something, it can be anything.

It is conventionally a place where declarations are made, but not implemented. If you're putting actual implementation in a .h file, then you may as well just call it a .c file.

The preprocessor and compiler don't care what the file extensions are, you can name them whatever you want. You can #include an .xls if you want - whether or not it will play nice with the compiler only depends on whether the contents are C-compliant.

So what I'm asking is what makes it so they can't implement stuff 100% in the form of code instead of having to use libraries.

That is what source code is. You are free to include the source code directly in your program (license permitting). You will likely run into build problems because this is not a supported use-case.

It would simplify everything for both parties.

Have some faith in the computer industry's best practices, in believing that no it wouldn't.

Now typically you'd separate the declarations in a .h and put the functions and actual work in .cpp or w/e but that's just details and it can work either way.

No this is not just details. The case where you have separated .h and .cpp requires linking in the same way that libraries require linking.

Consider a project with:

• foo.c
• foo.h
• main.c

where main.c includes foo.h

When you build this project the following things will happen:

1. foo.c will be compiled
2. main.c will be preprocessed. The line #include "foo.h" is literally replaced with the contents of foo.h.
3. main.c will be compiled. The compiler does not look at anything else when it does this.
4. main.c's object code will be linked with foo.c's object code.

When you have a library already, step #1 has already done for you - it is otherwise the same procedure.

for Linux in general

The concept of libraries and linking are on a basic level the same for Windows, Linux, OSX, Android, iOS, etc. It's important to know if you're writing software for any platform.

 

[Red Squirrel]

So any where I can find a tutorial that talks about all that? I'm looking for one specifically for Linux. Yes the concept may exist in other OSes but each one does it differently, I work mostly in Linux so I want to learn that setup specifically. Perhaps you are right and I need to learn how it works instead of trying to avoid it. To me the whole thing is just really convoluted but maybe a good tutorial will clear things up and make the troubleshooting process easier as I'll be able to know better how to deal with certain issues such as when it says it can't find a file even if the file does exist.

 

[Tawnos]

http://www.linux-mag.com/id/976/
http://www.cs.iit.edu/~iraicu/teaching/EECS211/handouts/EECS_211_What_is_Compiling_and_Linking.pdf
http://www.tenouk.com/Bufferoverflowc/Bufferoverflow1.html
https://gcc.gnu.org/onlinedocs/gcc-4.9.2/gcc/