nnxion

So what do you do if you want to push in some pointers to objects?
Say the object is 1 KB, you don't want to copy that 30 million times right?
What if you make a pointer to each of your objects and put that in your vector?

I think that's what the OP means.

__________________
"Employ your time in improving yourself by other men's writings, so that you shall gain easily what others have labored hard for."
-- Socrates

slyadams

This is what I do at the moment, I 'new' the object, then push the pointer.

Thing is, I am storing lots and lots of 'data points' consisting of 3 ints and a long. Thus, my object size is only 160bits and so a 32bit overhead to that is not insignificant.

I couldn't test the copy constructor yesterday because something came up, but will do today.

Thanks again everyone.

grumpy

That is normally the purpose of a cache. That doesn't mean the cache has to hold EVERYTHING though. For example, are there particular objects that are accessed frequently? Is there some other pattern to access (eg based on object X being accessed, is there a higher likelihood of some other object Y being accessed shortly afterwards)?
If the client has a 64 bit platform, odds are they won't be skimping on RAM anyway. One of the purposes of going 64 bit is that more RAM can be addressed. It might help if you asked the client how much RAM on the target machine can be assumed as a minimim. Then you can decide if you need to worry about memory usage or not.
All you really need to check is that the number of constructor calls matches the number of destructor calls. Keep in mind you need to track ALL constructors though.
A typical implementation of a list is as a linked list. If that's true, it means each node of the list contains your object, plus a pointer to another node in the list.

slyadams

Yep mate, know about caches and linked lists and the like.

The cache doesn't hold everything, trust me! It stores the last 50 days worth of data (there is over 2 years in the database). If a front end process requests data older than the 50 days then the cache will go and get this from the database, with an obvious performance hit. Pretty much all the data in the cache will be accessed regularly, due to the nature of the system.

While it may be true that the client won't be skimping on memory, it is still true that on a 64 bit platform, the pointer to the object will be 64 bits, while the object itself is only 160bits. To me that overhead is too high (30% of the space requirement for the data points would be in storing the pointers!).

We built the target machine and it is dedicated, pretty much, to running this process. The machien has, I think, 4GB of memory in it. Now, the reason we decided to use a 64bit machine for this installation is that the system deals with a LOT of unix timestamps (both in the database and the cache) and so we thought we'd see how much of a performance benefit we might get from getting a 64 bit CPU to handle this, rather than a 32bit CPU.

grumpy

My point is: WHY do you consider that overhead too high? If you are busily saving memory resources you don't need to you are just wasting effort.
Given you are controlling both the hardware and the software, it would be easier to make sure the hardware has enough memory. With 160 bit structs, 4G is more than enough. A Gig of RAM costs no more than (and probably less than) the time you've spent worrying about the memory usage of different techniques.
With that sort of rationale, I suggest doing the minimum needed to get the program running correctly on the 64 bit system, and then test your little heart out. I'd only worry about memory usage if testing showed it was a significant issue.

DaWei

I feel compelled to suggest that you study economics as well as C++. You can find bosses that don't know (or care) that you're tanking the bottom-line, but it's a chancy thing.

__________________
Abstraction doesn't make it impossible to write bad code; it makes it possible to write superior code.
Contributor's Corner: Grumpy on C++ Exceptions DaWei on Pointers

slyadams

The implementation is on a fixed price contract and currently we are undergoing a process to review the system components, and we have budget to do this. The issue here is the fact that this process is part of a commercial piece of software that we install for a number of clients.

Maximising the amount of history that can fit in an instance of the cache is very important because queries that request data from before the start are commonplace. Thus, if we can increase the history of the cache by 30% (e.g. 50 days -> 65 days) then this is a tangible benefit to at least one of our clients, who are prepared to spend money on our time, but are very resistent to spend a few hundred bucks on some more RAM. Telling you that its a 'government' client will tell you why.

I spent a couple of hours today modifying the code and checking out the constuctors, destructors and copy constructors and it all seems to work.

Also, the figure of 30m points was for a different client than the 64bit machine. The amount of points for this new client is, at the moment, not completely defined by the client. This is why it is important to us to maximise the amount of history that can be loaded.

DaWei

Well, nothing you can do about that, but pay more taxes. I sympathize, did tons of work for the USPS and some for the Air Force and State Department.

__________________
Abstraction doesn't make it impossible to write bad code; it makes it possible to write superior code.
Contributor's Corner: Grumpy on C++ Exceptions DaWei on Pointers

slyadams

Cheers mate. I'm pretty sure I'm there now. I've removed the pointers, created the correct copy constructor and done a bunch of burn tests and its stable and the internal 'memory counter' is consistent. All I need to do now is check that the code that creates and deletes subtrees still works, although I don't see how it won't.

Many thanks again