Programming Forums > Application Development > C++

slyadams · Mar 28th, 2006, 7:26 AM

Chaps,

I have s system where I am pushing objects into a vector inside a function. At the moment I 'new' the object and push the pointer and it works fine.

I was thinking that I might be able to save some memory space by pushing the object itself and not the reference. i.e.

currently:

void func1() {
vector.push_back(new ClassName(param1,param2));
}

possible:

void func1() {
vector.push_back(ClassName(param1,param2));
}

The problem is that the destuctor for ClassName gets called everytime that func1 returns in the 2nd example. The problem is that the object in the 2nd example is static and its scope is limited to the function func1.

So my question is: is there anyway to push objects dynamically into a vector without using pointers. The only reason I don't want to use pointers is that these vectors are at the bottom level of a tree structure and in a standard instance of the process will, in total, store about 30million objects. Thus, storing the pointer, as well as the data 'wastes' about 100meg. We are moving the process onto 64bit platforms for clients, so this overhead could, potentially, double!

Thanks in advance for any help.

P.S. :banana:

grumpy · Mar 28th, 2006, 7:40 AM

I suggest you read the sticky post on "How to ask a question". It will give you tips on how to ask questions to increase your chances of getting a useful answer.

I'm assuming the vectors in your two versions of func are of different types (eg the first is a std::vector<ClassName *> and the second is a std::vector<ClassName>).

The push_back() function of std::vector appends a COPY of the value it receives to the end of the vector. It doesn't matter whether the value it receives is a pointer (as in your first example) or an instance of a class (as in your second example), as long as the vector is of an appropriate type. All that is required to work with an instance of a class is that your class has a copy constructor and associated assignment operator that work as expected.

If you use a std::vector<ClassName *> and push_back new'd objects, you need to delete all objects in the vector before allowing the vector to be destroyed or pass out of scope. For example, the following are functionally equivalent (except the first works with dynamically allocated objects being pushed, and the second doesn't;

(Toggle Plain Text)

#include <vector>

std::vector<ClassName *> myvec;

void func()
{
    myvec.push_back(new ClassName(param1, param2));
}


int main()
{
     func();
     //  delete the object pushed by func() before allowing the vector to be destroyed
     delete myvec[0];
     // myvec will pass out of scope and be released
}

vs

(Toggle Plain Text)

#include <vector>

std::vector<ClassName> myvec;

void func()
{
    myvec.push_back(ClassName(param1, param2));
}


int main()
{
     func();
     // myvec will pass out of scope and be released, as will its elements
}

PS Children will insist on playing with bananas

slyadams · Mar 28th, 2006, 8:19 AM

OK, but in the first example there is an increased memory overhead isn't there? Won't the Object be created in memory somewhere and the pointer pushed into the vector? Thus there is a memory overhead increase of sizeof(void *)?

Polyphemus_ · Mar 28th, 2006, 8:30 AM

Quote:

Originally Posted by slyadams

OK, but in the first example there is an increased memory overhead isn't there? Won't the Object be created in memory somewhere and the pointer pushed into the vector? Thus there is a memory overhead increase of sizeof(void *)?

True, but you will notice the 4 bytes of overhead (at least on most 32 bit systems) is worth it.

slyadams · Mar 28th, 2006, 8:38 AM

The only problem is that when I've got my tree structure fully populated we're talking in the order of 30million nodes and so around 100meg of extra overhead.

But pushing the objects in directly I get 30 million calls to the class destructor. Which actually might not be too bad. Increased processing overhead, but all the destructor does it modify some variables that monitor the internal size/state of the process.

Polyphemus_ · Mar 28th, 2006, 9:40 AM

Well, when pushing 30 million of objects 100 mB isn't that much in comparison (well, at least I guess so, unless one object is 2 bytes big). If you don't need pointers in this situation, you don't have to, remember that. When using tree structures, you often use inheritance etc. though (I don't know if you are using it) and you will need to use pointers.

I'm trying to make clear that the use of pointers and dynamic memory allocation depends on the situation.

slyadams · Mar 28th, 2006, 9:59 AM

Yeah, the object sizes are only 160bytes, so an extra 4 bytes (8 bytes on the 64bit machines) isn't the end of the world, but would be nice to save.

I will test further just pushing the object itself in. Is there anything else I need to watch out for, apart from the fact that the destuctor will get called for the class for every push?

The Dark · Mar 28th, 2006, 6:11 PM

The reason the destructor is called when you push the object is, as grumpy said, that when you push the object the vector class takes a copy of the object, the original object that you created to push is still around and will be destroyed at the end of the function call.

If you are pushing a lot of objects into a vector, be aware that when the vector needs to increase its size, it may end up having to allocate a new block of memory, big enough to hold all the objects and then copy all the existing objects from the old memory block to the new one. This can be disastrous in terms of CPU time and memory use (for an amount of time it will have twice the memory used).

You can get around this problem either by using the reserve() member function for a vector, or by using the deque class instead.

If you do end up using pointers to objects in your vectors, making individual new calls will add an overhead to your memory needs, as each object allocated will need to have housekeeping data such as the size of the object stored alongside it. You may be better off allocating "chunks" of objects using new[] or even allocating objects using a few vectors and getting the pointers back from there.

grumpy · Mar 29th, 2006, 12:42 AM

OK; technically, if you are dynamically allocating the objects, there is an overhead of storing the pointer to it somewhere. As a std::vector typically allocates more memory than it needs (so, for example, it doesn't have to reallocate every time you add an object to it) the difference is usually insignificant. As The Dark said, you can use reserve() to work around that. In your example, if the vector reserves one element more than your required size, that's 160 bytes or the equivalent (on a typical 32 bit system) of 40 pointers.

Given that your collection of 30 million objects will take (roughly) 5 GB, a difference of 120MB in memory usage is insignificant ---whether you save it or not.

Yes, the vector<ClassName> invokes the ClassName destructor when you destroy the vector object. It also invokes copy constructor (or assignment operator depending on context) and the destructor when resizing. That doesn't happen with the vector<ClassName *> as a pointer has neither a destructor nor a constructor, BUT this approach guarantees you a memory leak unless you explicitly delete every instance.

I would describe what you're doing as a case of premature optimisation. It is better to implement your application as simply as possible, and be confident of it running correctly. In practice, for your application, I would actually prefer the vector<ClassName> approach. But NOT because it avoids the memory overhead of the vector<ClassName *> approach --- I would prefer it because it works correctly and you don't have to jump through hoops to avoid memory leaks. And I would accept the cost of constructor/destructor calls on resizing the array as a cost of having more chance of getting it to work correctly the first time.

Practically, I'd worry more about reducing the number of objects you need to keep in memory at any one time than I would about techniques to minimise the memory overhead of keeping 30 million objects in memory.

slyadams · Mar 29th, 2006, 2:32 AM

Thanks for your help guys!

Sorry, realised a made a blunder in one of my posts. 160bits is the size of the objects, not 160bytes. :o

I thought about ways to reduce the number of objects, but the problem is the data is storing rows from a database. This data is basically a 4 element data point (long,int,int,int) and the process needs to store loads of them to make them available for front end applications to query. Basically the process is a data cache and provides significantly quicker access than querying the database directly.

The process does work and has been in production for about a year now and I'm pretty sure doesn't have any leaks because the process is usually up for months on end, constantly reading new rows from the database and dropping old ones. We've just started an implementation for a new client who wants a 64bit platform. During this implementation I realised that the memory overhead of all these pointers will double.

I will write a copy constructor today and trace through the allocation and see what happens.

I will try and get around the vector resizing issue by looking at the impact of moving to a list. Although, the maximum length of a vector at the bottom of my tree is going to be 1440. With a object size of 160bits, this comes out at just under 30kB. Not that much at all, so maybe I don't need to worry about vector resizes.

Mar 28th, 2006, 7:26 AM	#1
slyadams Newbie Join Date: Mar 2006 Posts: 12 Rep Power: 0	Pushin an object into a vector Chaps, I have s system where I am pushing objects into a vector inside a function. At the moment I 'new' the object and push the pointer and it works fine. I was thinking that I might be able to save some memory space by pushing the object itself and not the reference. i.e. currently: void func1() { vector.push_back(new ClassName(param1,param2)); } possible: void func1() { vector.push_back(ClassName(param1,param2)); } The problem is that the destuctor for ClassName gets called everytime that func1 returns in the 2nd example. The problem is that the object in the 2nd example is static and its scope is limited to the function func1. So my question is: is there anyway to push objects dynamically into a vector without using pointers. The only reason I don't want to use pointers is that these vectors are at the bottom level of a tree structure and in a standard instance of the process will, in total, store about 30million objects. Thus, storing the pointer, as well as the data 'wastes' about 100meg. We are moving the process onto 64bit platforms for clients, so this overhead could, potentially, double! Thanks in advance for any help. P.S. :banana:

Mar 28th, 2006, 7:40 AM	#2
grumpy Programming Guru Join Date: Jun 2005 Location: Adelaide, South Australia Posts: 1,198 Rep Power: 5	I suggest you read the sticky post on "How to ask a question". It will give you tips on how to ask questions to increase your chances of getting a useful answer. I'm assuming the vectors in your two versions of func are of different types (eg the first is a std::vector<ClassName > and the second is a std::vector<ClassName>). The push_back() function of std::vector appends a COPY of the value it receives to the end of the vector. It doesn't matter whether the value it receives is a pointer (as in your first example) or an instance of a class (as in your second example), as long as the vector is of an appropriate type. All that is required to work with an instance of a class is that your class has a copy constructor and associated assignment operator that work as expected. If you use a std::vector<ClassName > and push_back new'd objects, you need to delete all objects in the vector before allowing the vector to be destroyed or pass out of scope. For example, the following are functionally equivalent (except the first works with dynamically allocated objects being pushed, and the second doesn't; (Toggle Plain Text) #include <vector> std::vector<ClassName > myvec; void func() { myvec.push_back(new ClassName(param1, param2)); } int main() { func(); // delete the object pushed by func() before allowing the vector to be destroyed delete myvec[0]; // myvec will pass out of scope and be released } #include <vector> std::vector<ClassName > myvec; void func() { myvec.push_back(new ClassName(param1, param2)); } int main() { func(); // delete the object pushed by func() before allowing the vector to be destroyed delete myvec[0]; // myvec will pass out of scope and be released } vs (Toggle Plain Text) #include <vector> std::vector<ClassName> myvec; void func() { myvec.push_back(ClassName(param1, param2)); } int main() { func(); // myvec will pass out of scope and be released, as will its elements } #include <vector> std::vector<ClassName> myvec; void func() { myvec.push_back(ClassName(param1, param2)); } int main() { func(); // myvec will pass out of scope and be released, as will its elements } PS Children will insist on playing with bananas

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Mar 28th, 2006, 8:19 AM	#3
slyadams Newbie Join Date: Mar 2006 Posts: 12 Rep Power: 0	OK, but in the first example there is an increased memory overhead isn't there? Won't the Object be created in memory somewhere and the pointer pushed into the vector? Thus there is a memory overhead increase of sizeof(void *)?

Mar 28th, 2006, 8:38 AM	#5
slyadams Newbie Join Date: Mar 2006 Posts: 12 Rep Power: 0	The only problem is that when I've got my tree structure fully populated we're talking in the order of 30million nodes and so around 100meg of extra overhead. But pushing the objects in directly I get 30 million calls to the class destructor. Which actually might not be too bad. Increased processing overhead, but all the destructor does it modify some variables that monitor the internal size/state of the process.

Mar 28th, 2006, 9:40 AM	#6
Polyphemus_ Expert Programmer Join Date: Aug 2005 Location: Rotterdam, the Netherlands Posts: 942 Rep Power: 3	Well, when pushing 30 million of objects 100 mB isn't that much in comparison (well, at least I guess so, unless one object is 2 bytes big). If you don't need pointers in this situation, you don't have to, remember that. When using tree structures, you often use inheritance etc. though (I don't know if you are using it) and you will need to use pointers. I'm trying to make clear that the use of pointers and dynamic memory allocation depends on the situation.

Mar 28th, 2006, 9:59 AM	#7
slyadams Newbie Join Date: Mar 2006 Posts: 12 Rep Power: 0	Yeah, the object sizes are only 160bytes, so an extra 4 bytes (8 bytes on the 64bit machines) isn't the end of the world, but would be nice to save. I will test further just pushing the object itself in. Is there anything else I need to watch out for, apart from the fact that the destuctor will get called for the class for every push?

Mar 28th, 2006, 6:11 PM	#8
The Dark Expert Programmer Join Date: Jun 2005 Posts: 810 Rep Power: 4	The reason the destructor is called when you push the object is, as grumpy said, that when you push the object the vector class takes a copy of the object, the original object that you created to push is still around and will be destroyed at the end of the function call. If you are pushing a lot of objects into a vector, be aware that when the vector needs to increase its size, it may end up having to allocate a new block of memory, big enough to hold all the objects and then copy all the existing objects from the old memory block to the new one. This can be disastrous in terms of CPU time and memory use (for an amount of time it will have twice the memory used). You can get around this problem either by using the reserve() member function for a vector, or by using the deque class instead. If you do end up using pointers to objects in your vectors, making individual new calls will add an overhead to your memory needs, as each object allocated will need to have housekeeping data such as the size of the object stored alongside it. You may be better off allocating "chunks" of objects using new[] or even allocating objects using a few vectors and getting the pointers back from there.

Mar 29th, 2006, 12:42 AM	#9
grumpy Programming Guru Join Date: Jun 2005 Location: Adelaide, South Australia Posts: 1,198 Rep Power: 5	OK; technically, if you are dynamically allocating the objects, there is an overhead of storing the pointer to it somewhere. As a std::vector typically allocates more memory than it needs (so, for example, it doesn't have to reallocate every time you add an object to it) the difference is usually insignificant. As The Dark said, you can use reserve() to work around that. In your example, if the vector reserves one element more than your required size, that's 160 bytes or the equivalent (on a typical 32 bit system) of 40 pointers. Given that your collection of 30 million objects will take (roughly) 5 GB, a difference of 120MB in memory usage is insignificant ---whether you save it or not. Yes, the vector<ClassName> invokes the ClassName destructor when you destroy the vector object. It also invokes copy constructor (or assignment operator depending on context) and the destructor when resizing. That doesn't happen with the vector<ClassName > as a pointer has neither a destructor nor a constructor, BUT this approach guarantees you a memory leak unless you explicitly delete every instance. I would describe what you're doing as a case of premature optimisation. It is better to implement your application as simply as possible, and be confident of it running correctly. In practice, for your application, I would actually prefer the vector<ClassName> approach. But NOT because it avoids the memory overhead of the vector<ClassName > approach --- I would prefer it because it works correctly and you don't have to jump through hoops to avoid memory leaks. And I would accept the cost of constructor/destructor calls on resizing the array as a cost of having more chance of getting it to work correctly the first time. Practically, I'd worry more about reducing the number of objects you need to keep in memory at any one time than I would about techniques to minimise the memory overhead of keeping 30 million objects in memory.

Mar 29th, 2006, 2:32 AM	#10
slyadams Newbie Join Date: Mar 2006 Posts: 12 Rep Power: 0	Thanks for your help guys! Sorry, realised a made a blunder in one of my posts. 160bits is the size of the objects, not 160bytes. :o I thought about ways to reduce the number of objects, but the problem is the data is storing rows from a database. This data is basically a 4 element data point (long,int,int,int) and the process needs to store loads of them to make them available for front end applications to query. Basically the process is a data cache and provides significantly quicker access than querying the database directly. The process does work and has been in production for about a year now and I'm pretty sure doesn't have any leaks because the process is usually up for months on end, constantly reading new rows from the database and dropping old ones. We've just started an implementation for a new client who wants a 64bit platform. During this implementation I realised that the memory overhead of all these pointers will double. I will write a copy constructor today and trace through the allocation and see what happens. I will try and get around the vector resizing issue by looking at the impact of moving to a list. Although, the maximum length of a vector at the bottom of my tree is going to be 1440. With a object size of 160bits, this comes out at just under 30kB. Not that much at all, so maybe I don't need to worry about vector resizes.

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)

Programming Forums > Application Development > C++

Pushin an object into a vector