Archive for January, 2011

Pointer Promises – what kind of things can we state?

Posted in cpp, Thinking Out Loud on January 30, 2011 by Crazy Eddie

Blog note: “Thinking Out Loud” categorized posts are little more than brain dumps that I am using to go through a thought process. Sometimes it helps to “talk” to someone in order to get your thoughts in some kind of order. That person doesn’t have to be real, they can be imaginary or, in this case, the general public who may or may not be remotely interested. Posting them does two things, allows me to get input from anyone so inclined and allows those who might be interested a chance to see what kinds of things someone like me thinks about when approaching a problem. So, take it for what it is.

Currently there are smart pointer libraries that express pointer scope and ownership but we have no kind of smart pointer types that express what you’re going to do with a pointer given to you. The closest two I can think of that might be considered to are auto_ptr and unique_ptr, simply because they assert that the object they govern can only be owned by one instance of their type. I ask you though, how might you write a function that asserts, in its interface, that the pointer being passed to it will be temporarily used, not copied, and that functions calling it are free to destroy the resource as soon as the function exits? I know of none.

We do have at least one precedent for desiring this kind of thing, the const keyword. You can use the const keyword as a member function modifier stating to all clients that, “This function will not alter the externally visible state of this object.” You can use the const modifier for reference and pointer parameters to state that, “This function will not attempt to alter the externally visible state of the object supplied as parameter.” Sure, it is given that you can break any of these promises but solid practices in C++ say that you both do not break such promises and that you do in fact make them so that clients can be aware of what you will and will not do as a side effect of your operations.

With all of that being said, I have decided that it’s time we had such a set of smart pointers. The question now is only what kind of statements do we want to express and what are the promises they make? Here’s the list of things I can think of that I might want to state about what I’ll do with a pointer supplied to me as a parameter:

  • I might try to make a copy of the object being pointed at.
  • I just need to temporarily use the object being pointed at, I’ll not retain reference or pointer to it.
  • I need to retain a pointer or reference to the object being pointed at, but I’m not going to take ownership of it.
  • I am going to retain a pointer or reference to the object being pointed at and need to retain sole ownership of it.
  • I am going to/may retain a pointer or reference to the object being pointed at and will share ownership of it with anyone else who is also retaining a pointer or reference.

At this point I’m of the opinion that the first statement isn’t really a promise about a pointer but a side effect of the pointed to object’s interface; thus it has no business being specified by a smart pointer library. The last two statements seem to already be taken care of by unique/auto_ptr and shared_ptr respectively. This leaves only the middle two.

The promise being made by the first of those two statements (type A ptr) would seem to me to be:

  1. I’m not myself going to make any copy of the pointer that is assigned to any variable that outlives the scope of this immediate call. and…
  2. I’m not going to pass this pointer to any function that does not make this same promise.

The promise being made by the second of those two statements (type B ptr) would seem to me to be:

  1. I need to be able to validate the validity of the pointer being referenced because I may outlive the object that owns it.
  2. I will never delete the object being pointed at by this pointer.
  3. I will not pass this pointer to any function that expects to hold ownership of it of any kind, shared or otherwise.

This would imply that you can pass one of the latter pointers to a function accepting one of the former, but not visa-versa. In other words, if you promise to not keep a reference to the object then you can’t pass to a function that doesn’t make that promise, but if you haven’t made that promise you can pass the pointer to one that does. In order to complete the rule set, we need to analyze the promises made by the last two, the ones that are already being implemented by existing smart pointers.

If you’re a function that claims to steal ownership (type C ptr) then the following statements are being made:

  1. Once I’m done, you don’t have to delete this object.
  2. Once I’m done, you’d better NOT delete this object.
  3. I promise to delete this object before I’m destroyed.
  4. You need to ask me for a pointer to this object because I might have destroyed it already.

If you’re a function that claims to share ownership of the object (type D ptr) then the following statements are being made:

  1. You have guaranteed, continued access to the object after making a call to this function iff you’ve kept shared ownership as well.
  2. You may have continued access to the object if you request it from the object this function is a member of.
  3. You may have continued access to the object if some other object besides this one has retained shared ownership of it.
  4. Nobody can steal sole ownership of the object.

At this point we can decide what kind of expressions the various kinds of pointers can be used in:

  • A type ‘A’ pointer appears to be the most restrictive. It thus can’t be passed to anyone expecting any of the other kinds of pointer. It can only be used in a type ‘A’ expression.
  • A type ‘B’ pointer is less restrictive (allows the client to do more) but poses more requirements on the caller. It can be used in a type ‘A’ expression and/or type ‘B’ expression but neither a type ‘C’ nor a type ‘D’ since they claim to have ownership responsibilities.
  • A type ‘C’ pointer takes ownership from the caller. It does not claim that it will never give it up nor destroy the pointer immediately. It can be used in type ‘A’, ‘B’, and ‘C’ expressions. It can give up ownership in a type ‘D’ expression.
  • A type ‘D’ pointer shares ownership. It can only destroy the object if it’s the last one left, which clients don’t know. It can be used in a type ‘A’ expression because they don’t make copies. It can be used it a type ‘B’ expression because they don’t take any kind of ownership. It can be used in another type ‘D’ expression because ownership is shared. They cannot be used in a type ‘C’ expression because they are not free to give up ownership.

We are left now with two issues left unexamined: raw pointers, references, and pointer returns. Let’s examine the first two together and leave the last for later. It would seem that examining the first two might give us some insight into the last.

The system being proposed would really be best served and used if neither raw pointers nor references even existed. Nothing can really be said about the former and use of the latter provides no guarantees either even if we are left with some extra requirements. Thus the best answer would seem to never, ever, allow converting raw pointers or references into smart pointers, or visa-versa. Unfortunately, this is impractical and ultimately impossible to implement. The second best answer is to provide explicit casts to/from smart pointers to the raw equivalents. The developer and team can then best decide whether to allow raw pointers at all and when/if to use references.

My advice would be to disallow the retaining of pointers or references to any kind of reference parameter. A reference could then be considered a type ‘A’ pointer or expression. It might, in fact, make sense to implicitly allow such use. Even more interesting, we might consider using a policy to differentiate between cases when any of the above pointer types can be NULL since this is one of the common reasons to use references to begin with (you can’t create a null reference within defined behavior). Even if we totally remove the need for references in this manner though there’s still other considerations such as legacy libraries and maybe even performance issues that I’m not fully aware of.

We do know though that a reference cannot form a type ‘C’ or type ‘D’ pointer. You cannot transfer ownership of a reference. You could pass a reference to a pointer that you could then take the address of and steal ownership of IT, but this is an inherently unsafe system that is prone to developer error. The whole point of this library is to avoid developer error (otherwise why even have smart pointers at all?) and to make it easier to use things the right way, and more difficult to use them the wrong way. Thus it would certainly make no sense to provide any type of conversion, implicit or otherwise, from a reference to a type ‘C’ or ‘D’ pointer.

The only one remaining is the type ‘B’ pointer. In this case it IS useful to use a reference as such a pointer, since ownership is not being transferred, but a reference doesn’t have enough information within it to implement one of the proposed requirements of the incoming data: namely that we can check that it’s valid before trying to use it. However, it certainly makes complete sense to pass a member variable to a function that claims it will retain a pointer to that variable but will not attempt to destroy it. What we need then is a method by which we can express this kind of use and none of our existing objects would appear to so far.

It could be argued that a shared_ptr system can be used to implement the terms of a type ‘B’ expression or pointer. Any object that holds a member can declare that member as a shared_ptr instead of a basic member and can then pass it off to functions that expect a weak_ptr parameter. However, there are two issues with this:

  1. We can’t use basic member variables for a type ‘B’ expression so implemented without hacks like supplying a “null-op deleter” to the shared_ptr. This is of course extremely dangerous and completely breaks the semantics of the shared_ptr construct.
    We can’t stop anyone from taking shared ownership of the data. In many cases this doesn’t matter much, but in others it indicates a serious error if information about an object outlives the object itself.

Noticing the fact that the shared_ptr construct in fact implements all the requirements of a type ‘B’ pointer though is not a useless comparison. We know that we can implement a type ‘B’ pointer in terms of a shared_ptr, we just need to add further restrictions that also allow us to use things like a “null-op deleter” safely. These facts may allow implementing a type ‘B’ pointer to be very quick and easy.

The final issue then regarding promises is return types. What are the various terms we would like to express when providing access to information about the state of a given object? What kind of information do we need to provide clients when handing out pointers? I can think of the following:

  1. I am giving up ownership to this pointer right now. You will either take that ownership over or the data being pointed at will be instantly destroyed.
  2. I am giving access to a pointer I may or may not be retaining some amount of interest in. I am thus going to give shared ownership of it so that others may keep it if they desire.
  3. I am insisting that I retain sole ownership of this data but am providing access to it.

These of course obviously coincide with type ‘C’, type ‘D’ and type ‘B’ pointers respectively. The first two can be implemented by the exact same objects, the last needs to be implemented by something special. You might wonder now why there is no type ‘A’ equivalent in this list; I propose that it makes no sense. The return of a function MUST express what kind of ownership is being retained by the object being called. A type ‘A’ expression does not indicate this kind of information, it only stipulates that there is no interest in ownership whatsoever.

All of this of course totally fails to address issues like multi-threading. It seems to me that such issues are best resolved with policies. A non-thread safe implementation of course could not be used without explicit cast in a thread-safe context but visa-versa would be fine. A thread safe interface would require locks though and objects like pointers to owned members would pose interesting dilemmas. Unfortunately, my experience in this area is literally non-existent; I think I’ve written maybe two programs that even USED threads and that’s far from enough to familiarize yourself with the various problems. Perhaps at a later date I’ll try to consider how MT would effect the various constructs.

Static keyword is back!

Posted in C++0x on January 18, 2011 by Crazy Eddie

According to the standard defects report, using the static keyword to declare internally linked entities has been undeprecated. If you did not know, use of the static keyword in this manner was declared deprecated in Annex D.2 in lieu of using unnamed namespaces (

In case you’re confused…

// somefile.cpp

// deprecated version
static int x = 42;

// "better" version
namespace { int x = 42; }

Of course, this deprecation was apparently voted out clear back in November and I’m only now becoming aware of it, but all I’ve got to say is, “About fucking time!” Every time someone told me that I should use anonymous namespaces instead of static variables I’ve had to hold back the urge to yell, “WTF is your malfunction??” The two constructs do totally different things and have totally different purposes. The only time and reason in which anonymous namespaces where better was when you needed a “private” entity that you could use in templates (since they require external linkage). Beyond that, they just gave you more to type and provided external linkage to things you don’t WANT to be externally linked.

Pardon me, but I feel a slight vindicated right now :p

Of course, I’ve never run into a case where it really mattered all that much (except wrt to the template thing) so I capitulated and started using anonymous namespaces instead of static, internally linked variables. It wasn’t like any vendor was going to actually remove the keyword, but in an effort to be “more standard” and use the supposedly “better way” I did it anyway. Now that even the standard committee has decided that this was a dumb move though, I might just go back.

Hello world!

Posted in Template Metaprogramming on January 16, 2011 by Crazy Eddie

Yes, you can.

#include <boost/mpl/string.hpp>
#include <boost/mpl/insert.hpp>
#include <boost/mpl/end.hpp>

namespace mpl = boost::mpl;

template < typename S1, typename S2 >
struct concat 
  : mpl::insert
    , typename mpl::end<S1>::type
    , S2

typedef mpl::string<'hell', 'o'> hello;
typedef mpl::string<'worl', 'd!'> world;

typedef concat
  , mpl::string<' '> 
  , world
>::type because_i_can;

#include <iostream>
int main()
  std::cout << mpl::c_str<because_i_can>::value << std::endl;