The Nightstar Zoo

Nightstar IRC Network - irc.nightstar.net
It is currently Sun Dec 17, 2017 6:34 pm

All times are UTC - 6 hours [ DST ]




Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Mon Jul 12, 2004 3:00 pm 
Offline
Energizer Bunny
User avatar

Joined: Wed May 22, 2002 12:24 am
Posts: 1634
So I was having trouble with some code I wrote (the geodesation code) while trying to run it on VS.NET.

After a lot of work, I finally figured out the problem:

I have this object (a triangular array templated object, it uses new[]) and I have to return it, complete with its original pointer, to the calling function. The problem? Well, the compiler sees this object dropping out of scope (despite it being returned), calls its destructor, and in doing so clears the memory underneath the pointer... which I need for the object to make any sense after the function returns. And then when it reaches the end of the calling function, the second object drops out of scope, and tries deallocating the memory again. This of course is an error.

How can I avoid this?

Vorn


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jul 12, 2004 4:03 pm 
Can't say anything for sure without seeing the code, but my guess is you need to define a copy constructor. The original object is going out of scope. What's returned is a copy of the object, and if you don't define a copy constructor CTheClass::CTheClass(CTheClass&); then the compiler defines one for you.

The default copy constructor does a bitwise copy of the data, which in this case, is not good, because it's bitwise-copying all of your pointers. All of the new object's pointers are the same as the old ones, and point to the same data. When the first object goes out of scope, that data is destroyed and you have invalid pointers.

In your copy constructor you'll need to allocate new storage for all of the pointed-to data, and then copy the data to the new storage.

This is also an example of why it's desirable to always deallocate storage in the same scope as it was allocated, but I'll hold off on the commentary without seeing your code.


Top
  
 
 Post subject:
PostPosted: Mon Jul 12, 2004 4:50 pm 
Hmm, If you're heap-allocating it with new, it shouldn't go out of scope. But if you're allocating it as a local variable in stack, it will go out of scope.

I just tried in C# (you didn't specify which language you were using...) and it wouldn't even allow me to allocate it in stack. I tried every variation I could think of... This code below functioned properly, and didn't deallocate the array.

Code:
      static int[] blah()
         {
         int[] blargh = new int[50];
         blargh[5] = 5;
         blargh[6] = 1;
         return blargh;
         }
      static void Main(string[] args)
         {
         int[] bar =   blah();
         Console.WriteLine( bar[5].ToString() + " " + bar[6].ToString());
         }



Top
  
 
 Post subject:
PostPosted: Mon Jul 12, 2004 7:40 pm 
Offline
Energizer Bunny
User avatar

Joined: Wed May 22, 2002 12:24 am
Posts: 1634
The code, heavily stripped:
Code:
template <class T> class MyArray {
private:
  T[] array;
public:
  MyArray(unsigned int size) {new T[size] array;}
  ~MyArray{delete[] array;}
}

MyArray<blah> foo(wooga, whooga, smooga){
  MyArray<blah> retval;
  ... // manipulate retval
  return MyArray;
}

int main() {
  MyArray<blah> thatsillyretval = foo(spleen, pancreas, isles_of_langerhans);
}


Vorn


Top
 Profile  
 
 Post subject:
PostPosted: Mon Jul 12, 2004 8:12 pm 
Copy constructor.

Code:
(inside the MyArray definition)

public:
  // Need to store the array size
  size_t ArraySize;
  MyArray(const MyArray& theOtherArray) {
    array = new T[theOtherArray.ArraySize];
    memcpy(array, theOtherArray.array, sizeof(T) * theOtherArray.ArraySize);
  }


Top
  
 
 Post subject:
PostPosted: Mon Jul 12, 2004 11:28 pm 
Offline
Nightstar Graveyard Daemon
User avatar

Joined: Mon Jun 03, 2002 8:30 pm
Posts: 1071
Location: Wouldn't you rather observe my Velocity?
Vorn the Unspeakable wrote:
Code:
  MyArray<blah> thatsillyretval = foo(spleen, pancreas, isles_of_langerhans);


Ask yourself, "What am I really doing here?"

Well, in the code, I mean. Not like a philosophical/existential question or anything.

foo creates a MyArray<blah> object out of a spleen, a pancreas, and an isles_of_langerhans.

It's a constructor.

Code:
template<class T>
MyArray<blah>::MyArray(spleen, pancreas, isles_of_langerhans) {
    // create MyArray object out of these bits.
}


Now, if for some reason your foo function may not be used directly as a constructor, consider writing it as a factory function.
Code:
template<class T>
static MyArray<blah>* MyArray<blah>::CreateArray(spleen, pancreas, isles_of_langerhans) {
    MyArray<blah>* pMyArray = new MyArray<blah>;
    // diddle with pMyArray;
    return pMyArray;
}


You may choose to write it not as a class function at all if you can do all the diddling you need to do without touching MyArray's privates; either way the key difference is that I'm not returning MyArray object, I'm returning a pointer to that object. The compiler already has a copy constructor set up for a pointer, and that's what gets copied out. The created object stays on the heap untill collected.

The caller assumes the responsibility for getting delete() called. You could write a smart pointer class for that if you really needed it, I guess. My money's on the regular constructor, though--I haven't seen how you use your code, but my reflex is to try to angle the code that direction naturally.

Failing that, using a pointer and/or a deliberate factory function are common idioms. (Doesn't make them any more valid, but there you go.)


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 13, 2004 5:04 pm 
Chalain wrote:
if you can do all the diddling you need to do without touching MyArray's privates


Tee hee!


Top
  
 
 Post subject:
PostPosted: Tue Jul 13, 2004 6:00 pm 
Offline
Nightstar Graveyard Daemon
User avatar

Joined: Mon Jun 03, 2002 8:30 pm
Posts: 1071
Location: Wouldn't you rather observe my Velocity?
gwalla wrote:
Tee hee!


Don't laugh. I have successfully taught people the C++ scoping rules in 30 seconds, as follows:
  • Public things are, well, public.
  • Parents let their children play in protected areas.
  • It's perfectly natural to touch your privates.
  • Your friends can touch your privates, too.
  • You children can NOT touch your privates, except maybe in some parts of Kentucky.


I had a manager come *this* close to sending me to sensitivity training for that last one. He relented when I pointed out that he'd never forget the scoping rules.

He laughed ruefully, and shook his head. "No, I guess I won't. But I kinda wish I could."

I get that a lot. :-)


Top
 Profile  
 
 Post subject:
PostPosted: Tue Jul 13, 2004 6:44 pm 
One downfall to Pi's method, if that's a really big, 2-4 gigabyte array, it's going to take a very long time to copy... They've actually made it very hard in C# to Copy things for this very reason. It can be bloody slow.

in C#, Python, etc, you're normally tossing around references to objects. It looks like you're passing around the objects themselves, but you aren't. That's inefficient. Python you do copy things way more than C# though. (i believe that a slice operator does a copy of the data. One old python idiom for copying a list was list2 = list[:])

If you want to mimic this behavior in C++, you want to realize that the only way to truly work with it as a reference is to make it a pointer. You can pass it as references with the & modifier in the argument list, but that's just window dressing for the pointer anyway.

Code:

  // Reformatted to whitesmith bracing style.
template <class T> class MyArray
  {
  private:
    T[] array;
  public:
    MyArray(unsigned int size) {new T[size] array;}
    ~MyArray{delete[] array;}
  }

MyArray<blah> * foo(wooga, whooga, smooga)
  {
  MyArray<blah> * retval = new MyArray<blah>;
  ... // manipulate retval

  return MyArray;
  }

int main()
  {
  MyArray<blah> * thatsillyretval = foo(spleen, pancreas, isles_of_langerhans);
  delete thatsillyretval;
  }


The other thing you'll note, is that arrays in C++ are normally pointers anyway. This would seem to trip alot of people up if you made something called an array that wasn't automatically a pointer...

All the more reason to use a different language. :D

Chalain wrote:
Failing that, using a pointer and/or a deliberate factory function are common idioms. (Doesn't make them any more valid, but there you go.)


Not any more correct, except its the way most of the sane languages do it. :) Silly C precompiler languages anyway. Of course, most of the sane languages also have garbage collection, so this may not make as much sense in a non-gc language. Making it a constructor of some sort seems to make sense in C++ if you're going for Stack-allocated objects.


Top
  
 
 Post subject:
PostPosted: Tue Jul 13, 2004 7:08 pm 
Kazriko wrote:
One downfall to Pi's method, if that's a really big, 2-4 gigabyte array, it's going to take a very long time to copy... They've actually made it very hard in C# to Copy things for this very reason. It can be bloody slow.

The difference between my "method" and the other posts in this thread is that I decided to answer the question rather than critique the design. As far as I'm aware, Vorn is required to implement this in C++ for a class, so suggesting different languages didn't seem terribly useful. Besides, Chalain addressed the design issues I would have brought up.

That said, yes, the pointer/reference method is definitely a better option and should be used for anything larger than few hundred bytes (and definitely for anything over your pagefile size (4kb-32kb)), though I would suggest that if you're working on 2-4Gb arrays in memory, and expecting anything to take less than a long time, you're probably three or four years ahead of your hardware. For working with most things over a few Mb, leaving it on the HD is generally a better option.

Kazriko wrote:
Of course, most of the sane languages also have garbage collection

Okay, this is the point where you apologize to C++ and acknowledge that you're bigoted and lazy because you can't write any code without safety rails. Don't make me come over there.

-Pi
-AKA MSmercenary, working in native C++ because DVD just doesn't play very well in managed code....


Top
  
 
 Post subject:
PostPosted: Wed Jul 14, 2004 1:30 pm 
Pi wrote:
Okay, this is the point where you apologize to C++ and acknowledge that you're bigoted and lazy because you can't write any code without safety rails. Don't make me come over there.

Aw. I wish I could have said it first. Ah well. I can say it too.

Kaz: What he said. :D

[rant]And let's not forget that garbage collection is almost invariably slower than doing it yourself. I can't tell you how many times I've heard "C++ will be obsolete in X years" said by someone who had never written a time-critical application. Hell, for that matter, assembly is still used, and that has even less protection than C++. You don't think people use that because it's pretty, do you? ;)[/rant]


Top
  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 12:50 am 
Raif wrote:
Pi wrote:
Okay, this is the point where you apologize to C++ and acknowledge that you're bigoted and lazy because you can't write any code without safety rails. Don't make me come over there.

Aw. I wish I could have said it first. Ah well. I can say it too.

Kaz: What he said. :D

[rant]And let's not forget that garbage collection is almost invariably slower than doing it yourself. I can't tell you how many times I've heard "C++ will be obsolete in X years" said by someone who had never written a time-critical application. Hell, for that matter, assembly is still used, and that has even less protection than C++. You don't think people use that because it's pretty, do you? ;)[/rant]


Heh, you must not have read my other rant.

There's nothing that you ever need to do that can't be done in either straight C or a GC language. C++ is an abomination against C. ;)

C is an almost frictionless language. Almost like python. It operates in a way that is intuitive if you think logically. C++ is NOT frictionless. Sometimes operating in almost bewildering strangeness on things that used to be simple in C. It used to have its place, as in more complex application programs. Once you have higher level languages that can meld seamlessly into C code, the place for C++ disappears, gobbled up by a combination of high and low level code. You end up with much safer code because most of the manipulation is done in a safe high level language, and fast code because the speed critical sections are in C.


Top
  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 3:26 am 
Kazriko wrote:
There's nothing that you ever need to do that can't be done in either straight C or a GC language. C++ is an abomination against C. ;)

There's nothing you ever need to do that can't be done in assembly. There's, in fact, nothing you ever need to do that can't be done in straight machine code.

Your argument is facetious. ;)

Quote:
C++ is NOT frictionless. Sometimes operating in almost bewildering strangeness on things that used to be simple in C.

"It's harder to learn so it can't be good." *snrk*

Quote:
It used to have its place, as in more complex application programs. Once you have higher level languages that can meld seamlessly into C code, the place for C++ disappears, gobbled up by a combination of high and low level code.

Right. This is an idea I've found to be quite neat and lovely in theory, but I've yet to see it go smoothly in practice. Fact is proper design makes your code inherently safe. Python and C# are implemented in C++, for Bob's sake.

Here's a wonderful example: Autopointers (a safe data type) are standard in C++. Nobody uses them. Why do you think that is? Further, if you really want GC, or safe pointers, you can build your own implementation and specialize it to your needs. You can't do that when the implementation was written for you.

As I said, it's dependent on good design. You're depending on a new layer (a managed language) to do that good design for you.

Now, to cut a long rant short, let me put it this way. You have your preference, and I have no problem with that. Don't try to pass it off as some uncontrovertible fact, however, and don't presume to think it works the same way for everyone.


Top
  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 10:49 am 
Offline
Nightstar Graveyard Daemon
User avatar

Joined: Mon Jun 03, 2002 8:30 pm
Posts: 1071
Location: Wouldn't you rather observe my Velocity?
Kaz, you make baby C++ cry. :cry:

Kazriko wrote:
There's nothing that you ever need to do that can't be done in either straight C or a GC language. C++ is an abomination against C. ;)


I'm surprised to hear you say this. I actually would have expected this out of Pi, not you. This is the Standard Argument Against C++ Raised By Hidebound Procedural Programmers<sup>*</sup>. I've seen you code Python, so I know you understand OO.

<sup>*</sup> Sorry about that, Pi.

OO runs counter to many features of C. Or rather, it imposes a bunch of structure that is orthogonal to the nature of C. This orthogonality is often viewed as "needless" to C programmers. More importantly, when C programmers venture into it thinking that "orthogonal to proceduralness" also means "I should be able to venture into objectland without impacting my efficiency" they end up getting bitten.

Now, I will concede this much: the C++ implementation of OO is actually quite poor. It doesn't hide the things it should well enough and it obscures some things it shouldn't too well. For example, strong typing is the tool of the devil. I *know* I'm in the minority on that view, but the reason C++ templates have gotten so arcane is because C++ botched the OO inheritance model with strong typing and a lack of reflection. But I digress.

Kazriko wrote:
C++ is NOT frictionless. Sometimes operating in almost bewildering strangeness on things that used to be simple in C.


There it is--procedural thinking isn't supposed to work in C++. It does, but you need to understand that when you do it, you're writing C, not C++, and you should avoid carelessly saying things in your code like "class" or "//".

I would submit that this is largely a misunderstanding of the C++ object model. Read Scott Meyers' Effective C++, Kayshav Dattatri's C++: Effective Object-Oriented Software Construction, and Coad & Nicola's Object-Oriented Programming and you'll be good to go. All three of these books were written prior to 1996, so it's not like programming well in C++ is a recent discovery. Coad & Nicola really hit the mark the closest, I think... they showed me that after coding C++ for 12 years, I was still trying to get away with mashing C concepts onto C++.

Kazriko wrote:
Once you have higher level languages that can meld seamlessly into C code, the place for C++ disappears, gobbled up by a combination of high and low level code. You end up with much safer code because most of the manipulation is done in a safe high level language, and fast code because the speed critical sections are in C.


Now THAT'S an interesting thought. C++ still provides more convenient integration with C than Python or Ruby. Both Python and Ruby, however, provide more powerful object models that C++ does. Give it 2 or 3 years, and see if interface writing becomes more seamless. I wrote C modules for Python two years ago, and it was an unmitigated bitch. Haven't written any for Ruby yet, but I predict the only difference will be that it will be an unmitigated, undocumented bitch.

(HAHAHA! Get it? See, it's funny because Ruby is notorious for lack of documentation, especially in English. Get it? Eh? EH? Oh, nevermind.)

You raise some valid points, Kaz, but I don't think the world is ready for you to win this argument yet. Most shops are still writing code in C++, which means that we haven't finished writing the legacy code we'll need to support for years after the language is dead. Worse, the general populace doesn't yet see a ready successor to C++, so the shops that are still writing in C++ don't see themselves as "still" writing C++. They see themselves as "current with the times". You could make this argument in as persuasively as possible, and at best, the response would be, "Yeah... so?"

I actually proposed at the original Coderfest that we write Ns20 in Python first, get it working, and then port each module to C++, keeping the system working as we went along. I still think it would have been a darn good idea, but Pi still gets this little twitch right here when I remind him I suggested it.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 2:35 pm 
Chalain wrote:
Now THAT'S an interesting thought. C++ still provides more convenient integration with C than Python or Ruby. Both Python and Ruby, however, provide more powerful object models that C++ does. Give it 2 or 3 years, and see if interface writing becomes more seamless.

As an addendum to this, that little voice in the back of my head is telling me that it thinks managed languages will REALLY catch on when optimizing compilers are writteen for them. :) Remember back when C++ rolled around a big barrier to adoption was because it was a pain in the ass to optimize ("It's too slow"). Now C++ is the fast language and Python et al need to catch up.

Quote:
I actually proposed at the original Coderfest that we write Ns20 in Python first, get it working, and then port each module to C++, keeping the system working as we went along. I still think it would have been a darn good idea, but Pi still gets this little twitch right here when I remind him I suggested it.

Interesting idea. I wouldn't have wanted to set it up, of course, but it would have been cool as a prototype.

So... does python incur any overhead when calling into C? (Do you take a hit for crossing that boundary too often?)


Top
  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 6:42 pm 
Raif wrote:
So... does python incur any overhead when calling into C? (Do you take a hit for crossing that boundary too often?)


It does it all the time. Alot of the built in functions in python are written in C... Some of the tips on optimising python tell you to use things like list comprehensions, because they run in C space rather than Python space. I'm not sure what kind of a performance hit you take by it, but apparently you take a larger one by doing a loop in Python than by calling out a C code and doing the same loop there.


Top
  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 7:18 pm 
Offline
Nightstar Graveyard Daemon
User avatar

Joined: Mon Jun 03, 2002 8:30 pm
Posts: 1071
Location: Wouldn't you rather observe my Velocity?
Everything in Python is an object, right? When you call into C, the C function gets handed a PyObject * for each argument, and must unpack it into the correct type. To do this, it has to call into some Python support libraries that knwo (for example) how to turn a PyObject into an int.

When you loop in Python instead of in C, every single loop involves this call into C which must then unpack the variables. Better to hand C a PyObject that can be unpacked (once) into an array of some data and let C do the looping. Dono why it's faster--seems to me it shouldn't be, since you gotta convert every array element--but it is.

Oh, and if your function returns data, you have to use the API to create a PyObject and return that.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 7:55 pm 
Chalain wrote:
Kaz, you make baby C++ cry. :cry:

Kazriko wrote:
There's nothing that you ever need to do that can't be done in either straight C or a GC language. C++ is an abomination against C. ;)


I'm surprised to hear you say this. I actually would have expected this out of Pi, not you. This is the Standard Argument Against C++ Raised By Hidebound Procedural Programmers<sup>*</sup>. I've seen you code Python, so I know you understand OO.


I started out on Atari Basic (if-then-goto programming) Went to Turbo pascal 4 (procedural), and learned Turbo pascal 7 (mediocre OO), Python (much better OO) and lisp (Functional) along the way. I really do prefer OO when it is done correctly. I also learned C (low level procedural) and C++ (bad-to-mediocre OO) in there. That said, C++ is an abomination on C, and an abomination on OO. ;) I'm not saying it's dead though. What I'm saying is that the niche that it filled can now be done better with combinations of other languages, and that new projects should probably not be done in it. They still can be though, and probably will be for some time. The whole security initiative may be what finally pushes C++ down into the same niches that C now occupies. (Embedded systems, kernels, drivers, time critical media code, etc.) It's just too easy to write horribly insecure code with C and C++, you need a special breed of programmers to write it.

Chalain wrote:
Kazriko wrote:
C++ is NOT frictionless. Sometimes operating in almost bewildering strangeness on things that used to be simple in C.


There it is--procedural thinking isn't supposed to work in C++. It does, but you need to understand that when you do it, you're writing C, not C++, and you should avoid carelessly saying things in your code like "class" or "//".


The problem is, C++ isn't ideal for OO, and isn't ideal for procedural either. As far as "//" goes, you're thinking too Ansi-C. When I say pascal, I typically mean "Turbo pascal" not "That purist variant of pascal that noone uses in real life" Same for C. I use C primarily for embedded systems. Mainly DynamicC which does NOT support classes. It does have //, because it's a useful extension that does not detract from the language's uses in embedded. You can do OO in C, but it's uncomfortable.

I should append "and is simple in any decent OO language" to the above. :)

Quote:
You raise some valid points, Kaz, but I don't think the world is ready for you to win this argument yet. Most shops are still writing code in C++, which means that we haven't finished writing the legacy code we'll need to support for years after the language is dead. Worse, the general populace doesn't yet see a ready successor to C++, so the shops that are still writing in C++ don't see themselves as "still" writing C++. They see themselves as "current with the times". You could make this argument in as persuasively as possible, and at best, the response would be, "Yeah... so?"


I guess my company is ahead of the times then. We have been doing a nice mixture of low level code and high level code for ages, but we sort of split it differently. We actually code the low level code (DynC, ASM, Ladder logic, Optoio) on small embedded systems, and have it communicate to the python, c#, php, and VB code on the actual PCs. (oh, and the VB code is slowly disappearing, being replaced my C#... Darn legacy code anyway. :)

We've only once had any reason to code any lower level than VB on a pc, and that was one time-critical application that was a little too beefy for our 8-bit embedded systems. Normally those 8 bit systems have all the speed in the world needed for any application we throw at them. (storage space is another matter, as is UI. that's where the high level code comes in.)

Also, I can see this changing in the next 5 years as MS starts cramming its security initiative down everyone's gullets. .Net already has it setup so you can mix low-level hard-to-write-hard-to-secure C++ modules together with Easy high level C# code.


Top
  
 
 Post subject:
PostPosted: Thu Jul 15, 2004 8:05 pm 
Chalain wrote:
When you loop in Python instead of in C, every single loop involves this call into C which must then unpack the variables. Better to hand C a PyObject that can be unpacked (once) into an array of some data and let C do the looping. Dono why it's faster--seems to me it shouldn't be, since you gotta convert every array element--but it is.

Yeah, my question was more in relation to C code. I know calling out to C is faster than doing it in python, but calling C to do an inner loop and dropping back out would be slower than just that inner loop in native C code. Judging by what you said, that overhead is significant.

Kazriko wrote:
What I'm saying is that the niche that it filled can now be done better with combinations of other languages, and that new projects should probably not be done in it.

This is actually why I asked the above question. If the overhead is as significant as it looks, calling into C code for the expensive operations will still cost you quite a bit (over doing it in C++). You really need to be able to compile python code together with C code (with minimal fuss) and optimize it for this to be truly viable for everyone. ;)


Top
  
 
 Post subject:
PostPosted: Fri Jul 16, 2004 12:44 am 
Raif wrote:
Chalain wrote:
When you loop in Python instead of in C, every single loop involves this call into C which must then unpack the variables. Better to hand C a PyObject that can be unpacked (once) into an array of some data and let C do the looping. Dono why it's faster--seems to me it shouldn't be, since you gotta convert every array element--but it is.

Yeah, my question was more in relation to C code. I know calling out to C is faster than doing it in python, but calling C to do an inner loop and dropping back out would be slower than just that inner loop in native C code. Judging by what you said, that overhead is significant.


I believe someone said 20 C instructions for every Python instruction... It could have been as high as 40. Not sure if that means a loop done in Python vs. done in a list comprehension would be 1/20th the speed, or not. Oh, they applied this to reduce, and sort as well. If you use a built in C instruction in reduce, it does the inner loop entirely in C, which makes it a bit faster.

Quote:
Kazriko wrote:
What I'm saying is that the niche that it filled can now be done better with combinations of other languages, and that new projects should probably not be done in it.

This is actually why I asked the above question. If the overhead is as significant as it looks, calling into C code for the expensive operations will still cost you quite a bit (over doing it in C++). You really need to be able to compile python code together with C code (with minimal fuss) and optimize it for this to be truly viable for everyone. ;)


Actually, judging by the fact that python is apparently 100% written in straight C, calling out to C++ would probably be more expensive than C.

And you don't need to compile the Python code, all you need to compile is the C modules. judging by pyao and pymad, this is really easy to do. (python config_unix.py ; python setup.py build ; sudo python setup.py install) It has 2 scripts that do it all, config_unix.py and setup.py. And these are a heck of a lot simpler than autoconf and automake which are the staples for compiling c/c++ programs in posix systems.


Top
  
 
 Post subject:
PostPosted: Fri Jul 16, 2004 6:06 am 
Kazriko wrote:
Actually, judging by the fact that python is apparently 100% written in straight C, calling out to C++ would probably be more expensive than C.

That's not what I'm suggesting. I'm suggesting that even if you write your app in python and port the slow bits to C, you'll incur overhead [i]around[i] those slow bits that may be unacceptable when compared with writing the whole thing in straight C++. I'm trying to show that only very specific facets seem to be taken into consideration.

It's analogous to calling a function (which has to call out, push the stack, etc) versus an inline (which sticks the code where it needs to be).


Top
  
 
 Post subject:
PostPosted: Sat Jul 17, 2004 12:45 pm 
Offline
Nightstar Graveyard Daemon
User avatar

Joined: Mon Jun 03, 2002 8:30 pm
Posts: 1071
Location: Wouldn't you rather observe my Velocity?
Kazriko wrote:
Actually, judging by the fact that python is apparently 100% written in straight C, calling out to C++ would probably be more expensive than C.


You are correct, but only at development time, not runtime. It requires the addition of the words

Code:
extern "C" {
...
}


to your file. Strictly speaking, the time required to type these 13 extra characters DOES make the project more expensive. ;-)

Once compiled, the only difference between C++ and C is that the name mangler has had a go at your function table. (And extern "C" tells the mangler to go mangle itself.)


Top
 Profile  
 
 Post subject:
PostPosted: Sun Jul 18, 2004 8:41 pm 
Chalain wrote:
Kazriko wrote:
Actually, judging by the fact that python is apparently 100% written in straight C, calling out to C++ would probably be more expensive than C.


Once compiled, the only difference between C++ and C is that the name mangler has had a go at your function table. (And extern "C" tells the mangler to go mangle itself.)


I think you missed my point. The point was, that writing a c++ class, then trying to adapt it to a python class would take an entire extra abstraction layer that wouldn't be there if the class was implemented as a python class in C instead of a C++ class in C++. The object models are just too different. And that is the only possible way I could see that a python object would be any easier to develop in C++ than C. This is where a C++ class call would be far more expensive than a C class.


Top
  
 
 Post subject:
PostPosted: Sun Jul 18, 2004 8:46 pm 
Raif wrote:
Kazriko wrote:
Actually, judging by the fact that python is apparently 100% written in straight C, calling out to C++ would probably be more expensive than C.

That's not what I'm suggesting. I'm suggesting that even if you write your app in python and port the slow bits to C, you'll incur overhead [i]around[i] those slow bits that may be unacceptable when compared with writing the whole thing in straight C++. I'm trying to show that only very specific facets seem to be taken into consideration.


You're failing to take into account Programmer cycles.

Also, there are some clear examples of a Python/C project actually running faster than a similarly implemented C only project. Z2Medusa vs. Apache is one common example. Python lets you free yourself to work on the algorithm rather than muck around with the hardware, resulting in potentially more efficient algorithms. And if you've taken theory of algorithms at all, you should know that all the raw speed in the world can't make up for a bad algorithm.


Top
  
 
 Post subject:
PostPosted: Sun Jul 18, 2004 11:29 pm 
Kazriko wrote:
You're failing to take into account Programmer cycles.

Actually I'm not. :) When performance becomes a problem, developer cycles must be spent finding a way around it. Your suggestion only applies in situations where this problem of performance is outweighed by the time saved in design.

Also don't forget that converting older programmers to The New System also costs developer cycles.

Quote:
Also, there are some clear examples of a Python/C project actually running faster than a similarly implemented C only project. Z2Medusa vs. Apache is one common example. Python lets you free yourself to work on the algorithm rather than muck around with the hardware, resulting in potentially more efficient algorithms.

An efficient algorithm is wonderful, but an efficient algorithm in Python will never be faster than the same efficient algorithm in C(++) when both are done properly. ;)

If I'm reading you correctly you're saying, "It's easier to do properly in Python." I submit that the gap isn't necessarily all that wide. ;)

Quote:
And if you've taken theory of algorithms at all, you should know that all the raw speed in the world can't make up for a bad algorithm.

But C++ doesn't preclude good algorithms, and Python doesn't preclude bad ones. This is a question of experience, not so much trouble with the language.

I'd like to stress that not everyone has enormous trouble with C++. Some (myself among them) consider it to be relatively intuitive. Keep in mind that I'm no stranger to Python, and I'm certainly no stranger to C++. Also remember that C++ has some mighty powerful API's at its disposal which embody both high speed and good design.


Top
  
 
 Post subject:
PostPosted: Tue Jul 20, 2004 8:09 pm 
Raif wrote:
Kazriko wrote:
You're failing to take into account Programmer cycles.

Actually I'm not. :) When performance becomes a problem, developer cycles must be spent finding a way around it. Your suggestion only applies in situations where this problem of performance is outweighed by the time saved in design.


First, Python's native speed is typically just fine. The majority of applications you write will need no optimization at all in their native python versions. The key is, that optimization option is there when you need it.

One big bonus in dev time for the Python model is that you have a working program much faster that can be delivered to the customer as a prototype despite potential speed problems. You also continue to have a working program through all the optimization phases if they are required. You drop optimized modules in one bit at a time.

Speed to market is probably far more important than the operating speed these days. Operating speed can usually be solved with an older 2-3 ghz processor. In my line of work, especially. We need to have these programs released yesterday, frequently less than 20 hours of coding time. Heck, one application we implemented, The PC side of it was coded in 5 hours with python and shipped to the customer as a test unit. We've now spent dozens of hours refining it into a fully gui c# application. We probably wouldn't have been able to get it to the customer in time if we had started in C++ at the beginning. (C# would have taken at least 12 hours for the original prototype.)

Quote:
Also don't forget that converting older programmers to The New System also costs developer cycles.


So, how long did it take you to learn python? I think I was able to write decent code in it in around a day...

Quote:
Also remember that C++ has some mighty powerful API's at its disposal which embody both high speed and good design.


Gee, API's which can be used in python as well. Kind of interesting, that. I managed to put an mp3 and ogg player together in an hour or so using python and one of these "mighty powerful API's" that were written for C...

I suspect it would have taken much longer had I actually USED C++ to develop it, and all I'd have gained is maybe a 1-1.5% lower cpu usage (which is already under 5% on my dev system, and my development system is actually much slower than the production system.)

You're still forgetting my main argument. C++'s old niche is now better taken up by other languages, and C++ is being squeezed into an ever smaller niche, mostly one its inheriting from C. (Game programming, for one.) This is actually the same niche that C inherited from ASM...

And really, C++ is just a terrible language once you've learned how OO really works. Not difficult or unintuitive, just terrible OO. More time consuming because you have to work around its quirks, and then you end up with utterly insecure output code unless you spend 50-100% more time searching for potential buffer overflows.


Top
  
 
 Post subject:
PostPosted: Tue Jul 20, 2004 8:58 pm 
Kazriko wrote:
First, Python's native speed is typically just fine. The majority of applications you write will need no optimization at all in their native python versions. The key is, that optimization option is there when you need it.

It's about 1/10th the speed of C, last I read, which means it can handle approximately 1/10th of an application that brings a modern machine to its knees (games for instance).

Quote:
One big bonus in dev time for the Python model is that you have a working program much faster that can be delivered to the customer as a prototype despite potential speed problems.

Agreed, completely.

Quote:
Speed to market is probably far more important than the operating speed these days.

For some applications. The bleeding edge is a bit more lawless. ;)

Quote:
Operating speed can usually be solved with an older 2-3 ghz processor.

This is actually factored in. You don't need a top of the line machine to run one of these killer apps well. It doesn't pay to only target the top of the line. This does place some restrictions on operation, however. Namely that you can't expect to have a fully capable PC at your disposal.

Quote:
We probably wouldn't have been able to get it to the customer in time if we had started in C++ at the beginning. (C# would have taken at least 12 hours for the original prototype.)

Agreed. Pure C++ is absolutely one of the worst languages for rapid application development. Borland C++ Builder mitigates that aspect of it a great deal (though that's not really C++). In fact, the guy in charge of that portion of C# was the same guy who came up with the Borland version. :) It's got a way to go yet, but it's getting there I'm sure.

Standard GUI's have long been recognized as one of C++'s worst weaknesses. I'm not disputing that. It's still not anywhere near the whole picture, though.

Quote:
So, how long did it take you to learn python? I think I was able to write decent code in it in around a day...

Not long. However, I still haven't learned how to properly integrate C modules into python. And I've certainly not learned a lot of the caveats.

Quote:
Gee, API's which can be used in python as well. Kind of interesting, that. I managed to put an mp3 and ogg player together in an hour or so using python and one of these "mighty powerful API's" that were written for C...

Don't get so smug. C++ has it. So does python. The fact that C++ has it mitigates a huge amount of the development time. It's not fair to assume (which is how I'm reading your comments) that C++ forces you to develop everything from scratch. Proper modularity will save a lot of that. Build it once, build it right, and re-use it.

Python saves a lot of that first step, because they've done it for you. It seems to me much like the reason people refuse to use the STL. It has to be correct, and generalized, which makes it slow. If you know you'll never really run into a specific case, and you know you'll need certain functionality often, you take the STL and write your own spin on it that's much more suited to your application.

Quote:
I suspect it would have taken much longer had I actually USED C++ to develop it, and all I'd have gained is maybe a 1-1.5% lower cpu usage (which is already under 5% on my dev system, and my development system is actually much slower than the production system.)

Again, you're not choosing an example of a high-load application. FMOD does sound beautifully. If I wanted to do that, I could throw together something of a simple GUI and such in very short order. The part that would normally take a lot of time (properly reading and playing audio), is already done for me by a library I can reference directly from C++ anyway. Short story: Bad example. :)

In fact, C++ would allow me to build a beautiful 3d visualization studio far superior to many of the ones out there with (probably) far less development time than would be required in Python. :) There is no OS-based GUI work involved, yet it would look far superior.

Quote:
(Game programming, for one.

I wouldn't call that a small niche. :) And yes, this is where C++ belongs, and this is where I'm arguing for it. Bleeding edge applications, which will always be around (as long as we have enough lack of power that we need hardware upgrades, we'll have the bleeding edge).

And, of course, where the bleeding edge has a need, there are libraries to fill (DirectX and OpenGL have both developed to a point where they're pretty intuitive to use). This means that not-so-bleeding-edge applications have something else to add to the tool box.

Quote:
And really, C++ is just a terrible language once you've learned how OO really works. Not difficult or unintuitive, just terrible OO.

That doesn't make it a terrible language (there are, in fact, a few threads around here which raise the excellent point that OO is not, in itself, necessarily desirable).

Quote:
unless you spend 50-100% more time searching for potential buffer overflows.

Ah, the wily buffer overflow. Cause of almost all C bugs. This has not happened to me in a couple years (as far as I can recall). It's been remarked by several that you'll only perpetrate a bug a few times. The first few times you'll learn how to fix it faster. Finally you'll code in such a way that it doesn't happen. Does this mean you can't slip up now and again? Certainly not.

However, remember that Python is written in a low level language. If they can avoid buffer overflows in their code, so can I (and it doesn't have to take me ridiculous amounts of time to do it).


Top
  
 
 Post subject:
PostPosted: Thu Jul 22, 2004 5:58 pm 
Raif wrote:

Quote:
So, how long did it take you to learn python? I think I was able to write decent code in it in around a day...

Not long. However, I still haven't learned how to properly integrate C modules into python. And I've certainly not learned a lot of the caveats.


I have yet to encounter an actual need for this. It's there if I ever need to, however.

Quote:
Quote:
Gee, API's which can be used in python as well. Kind of interesting, that. I managed to put an mp3 and ogg player together in an hour or so using python and one of these "mighty powerful API's" that were written for C...

Don't get so smug. C++ has it. So does python. The fact that C++ has it mitigates a huge amount of the development time. It's not fair to assume (which is how I'm reading your comments) that C++ forces you to develop everything from scratch. Proper modularity will save a lot of that. Build it once, build it right, and re-use it.


As far as my comments on C++, I was envisioning using the exact same library in C++, and it still taking far longer to develop.

You say this as if there aren't dozens of other details slowing you down in C++... Like having to work around its clunky object model and having to do things the exact opposite of the way you were envisioning it because of said clunky object model.

Quote:
Quote:
I suspect it would have taken much longer had I actually USED C++ to develop it, and all I'd have gained is maybe a 1-1.5% lower cpu usage (which is already under 5% on my dev system, and my development system is actually much slower than the production system.)

Again, you're not choosing an example of a high-load application. FMOD does sound beautifully. If I wanted to do that, I could throw together something of a simple GUI and such in very short order. The part that would normally take a lot of time (properly reading and playing audio), is already done for me by a library I can reference directly from C++ anyway. Short story: Bad example. :)


The majority of applications are things that would not normally take much processing time. And if needed, you can always optimize the ones that aren't down to C. Of all the applications we've done at my work, only one of them out of a couple dozen has needed any more speed than straight python provided. An MP3 player is a good example of an application with an average processor load compared to most real world applications. The thing that would really slow you down in C++ is creating the object structure around the libraries. Heck, the self registering decoder module manager (ogg/mp3/flac/etc) would be a nightmare to implement in C++. In Python, it was a 30 minute project.

Quote:
In fact, C++ would allow me to build a beautiful 3d visualization studio far superior to many of the ones out there with (probably) far less development time than would be required in Python. :) There is no OS-based GUI work involved, yet it would look far superior.


How do you figure? I can have access to all the same libraries you do, including OpenGL. Many with more intuitive interfaces than their C++ versions provide. Look superior? To what?

Quote:
Quote:
(Game programming, for one.

I wouldn't call that a small niche. :) And yes, this is where C++ belongs, and this is where I'm arguing for it. Bleeding edge applications, which will always be around (as long as we have enough lack of power that we need hardware upgrades, we'll have the bleeding edge).


Compare all the number of cutting edge 3d game programmers that actually have jobs in the field to all the other programmers around the world. I'd say it's pretty darn small. Judging by the fact that most cutting edge 3d games I've seen only have a limited number of programmers in comparison to all the artists, sound people, producers, and designers they employ. I think FF7 only had 5 programmers on listed in the credits compared to dozens of other staff members.

Even without that argument, it is a niche none-the-less. It's also a niche that continually moves up in languages, albeit at a slower rate than the much larger general purpose application domain. It may be another 10 years before it gets shoved out of that niche, or at least down to where only 1/100 games are written in it... (in the mid 90's, there were still some tiny number of games written in pure ASM (not counting portable games systems.) I suspect that C++ will still have about that many games around the mid 2010's.)

Quote:
Quote:
And really, C++ is just a terrible language once you've learned how OO really works. Not difficult or unintuitive, just terrible OO.

That doesn't make it a terrible language (there are, in fact, a few threads around here which raise the excellent point that OO is not, in itself, necessarily desirable).


The point is, if OO is desirable in your application, then other languages are probably much better. If OO is not desirable, then there are still more languages that are better for the application.

Quote:
Quote:
unless you spend 50-100% more time searching for potential buffer overflows.

Ah, the wily buffer overflow. Cause of almost all C bugs. This has not happened to me in a couple years (as far as I can recall). It's been remarked by several that you'll only perpetrate a bug a few times. The first few times you'll learn how to fix it faster. Finally you'll code in such a way that it doesn't happen. Does this mean you can't slip up now and again? Certainly not.


Except that C++'s standard libraries are designed in such a way that it is counter-intuitive to write buffer safe code. Sure, there are people who can write that way, but it takes a good deal of additional work to write it this way. It's not just a "oh, i'll add this one line here to prevent that bug."

There are people who have gotten writing buffer safe code down in C/C++... But they usually write their own replacement for the standard libraries and don't touch the standard lib at all themselves.

Quote:
However, remember that Python is written in a low level language. If they can avoid buffer overflows in their code, so can I (and it doesn't have to take me ridiculous amounts of time to do it).


Except that you're now going to have to do it on every line of code you write. In python, they only need to avoid the overflows in the python source itself, then every line of python code is then as buffer safe as they have made the libraries. Again, in C++, you have to make sure EVERY time you use one of these unsafe standard functions that you are properly allocating the buffer, and not exceeding that buffer at any point in your code. You also have to make sure you properly deallocate that buffer, and then make sure there aren't any other pointers laying around to it that might get used...


Top
  
 
 Post subject:
PostPosted: Thu Jul 22, 2004 8:11 pm 
I almost typed up a gigantic essay to respond to all of Kaz's points, each of which makes sense from a certain point of view (his), but then I realized that before I even read any more of the posts here, most of which appear to be going over the same points over and over, I need the answer to an important question:

Who are you trying to convince, Kaz?

I mean, there's a fine line between discussion and internet holy war, and I think this thread has stepped over it. Basically, I've made my points, Raif has made his, and judging by the repeats, you've made all of yours too. With the possible exception of trolling (a fun passtime), what reason is there to continue? Just to make sure the other person doesn't get the "last word"? Many a holy war (and forum topic) have fallen into that cycle and forgotten to stop. I don't want that to happen to this forum. Code Monkeys is not D&E. I still read Code Monkeys.

-Pi
-No longer playing the "my language is holier than yours" game


Top
  
 
 Post subject:
PostPosted: Thu Jul 22, 2004 9:18 pm 
Pi wrote:
Code Monkeys is not D&E. I still read Code Monkeys.

Good point. I was about to respond but I'll drop it. :)


Top
  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 35 posts ]  Go to page 1, 2  Next

All times are UTC - 6 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group