There are a number of reasons why you may want garbage collection/automatic memory management over RAII:
- RAII-style freeing means a lot of reference counting. Reference counting and multiple threads don't mix well. You can circumvent that by having multiple separate heaps, but that's not always a practical solution.
- Reference counting is slow; if you want GC-like speed and RC-like pause time guarantees, you're probably better off with deferred reference counting, but that's not easy to engineer into C++; it's essentially a form of garbage collection.
- Lack of fully automatic memory management may affect modularity adversely; see Jones/Lins for the gory details. In brief, wholly or partly manual memory management creates leaky abstractions (this is if you eschew RC for some of your code/types for speed reasons or because you need to deal with cyclic references).
- It is not difficult to have scoped deallocation for resources (scope statement in D, using statement in C#, etc., destructor pragma in Nimrod). Resource lifetime, in any event, does not always coincide with the lifetime of a variable, so this is an incomplete solution. Using GC does not mean not using RAII where its downsides do not matter.
RAII does not mean a lot of reference counting. In the vast majority of cases in typical programs, objects have exactly one owner, and managing that ownership is very easy if your language provides good facilities for move semantics (as C++11 and Rust do).
You can't say reference counting is bad for multi-threaded programs without mentioning that GC is also bad for multi-threaded programs, since the garbage collector has to pause all other threads, either for the entirety of the GC run or (if you're lucky and using a good GC) for parts of it.
The modularity argument is reasonable, and I've been annoyed by this aspect several times, though in the majority of cases it doesn't seem to be an issue.
If you mix up reference counting with manual memory management, you get the aforementioned modularity issue (as I mentioned as one of my points). You get rid of (some) of the overhead, at the cost of simplicity (you also may incur other overhead; manual memory management often introduces unnecessary copying in order to keep ownership simple).
The problem with reference counting in concurrent programs is one of correctness, not one of speed [1] because every change of a reference count is a potential race condition. That's quite different from the challenges garbage collection faces in a multi-threaded environment.
Whether a garbage collector has to pause all threads is an implementation issue. Modern garbage collectors can limit that pause effectively to the root scanning phase, then let the collector run concurrently with the mutator. You can also work around pausing threads entirely, though the tradeoffs are rarely worth it outside of hard realtime environments.
Note also that this is an issue of pause time, not performance. While HN can get obsessed with pause times, not everyone programs video games, embedded systems, or kernel drivers where that matters. High-performance computing, for example, is a huge field where pause times are all but irrelevant and where amortized cost is what matters. (HPC is also where correctness can become easily more important than squeezing out a percent or more of performance through low-level hacks; if a job that takes several days to run crashes halfway through, that's an enormous loss.)
[1] Technically, you can make reference count updates atomic, but the overhead is absurd, especially under contention. Hence why SNZIs [2] exist, which reduce the speed overhead at the expense of additional memory overhead.
[2] http://dl.acm.org/citation.cfm?id=1281106
>While HN can get obsessed with pause times, not everyone programs video games, embedded systems, or kernel drivers where that matters.
The (my) problem is that it also matters whenever a program holds lots of data in memory (databases, data analysis, caching, etc). As the cost of RAM decreases, the importance of this problem increases, i.e. fast.
Large heaps and large pause times do not necessarily go hand in hand. That's a question of GC technology. For example, the Azul GC does <10 ms pauses for heaps that are hundreds of GB in size. Granted, Azul/Zing pricing doesn't exactly make it the most accessible technology (and it relies on kernel hackery for its compaction), but it demonstrates that multi-second pauses are hardly a necessity.
Incremental garbage collection isn't a new technology [1] or one that's particularly difficult to implement by itself (a grad student could probably implement Baker's treadmill in a couple of days); what makes it hard is primarily compaction and multiple threads [1]. You can't effectively use (naive) RC with multiple threads and you don't get compaction with RC, either. Multi-threading is in practice a major driver for the need of GC, since (naive) RC isn't thread-safe and unpredictable object lifetimes don't go well with manual memory management.
Also, deferred RC strategies can under certain circumstances contain the cost for cycle collection. Trial deletion is already limited to nodes reachable from other nodes whose reference count has been decremented; type information can be leveraged further to exclude nodes that cannot possibly be parts of cycles (this is particularly easy in ML-style languages, which make mutually recursive types explicit in the source code, but is not limited to those [2]).
Finally, you can also use multiple disjoint heaps to simplify implementation and cap GC cost (one per thread and zero or more shared heaps). This can also bring naive RC back into play as a viable strategy, though you'd still lose compaction. Multiple heaps are particularly attractive for NUMA architectures.
[1] I note that hard realtime guarantees are difficult to make with a basic incremental collector due to the potential of pathological behavior, but we are not talking about hard realtime guarantees here.
[2] Obviously, JITs and dynamically or weakly typed languages have a harder time with this approach.
>Large heaps and large pause times do not necessarily go hand in hand
Not necessarily, but in practice they do go hand in hand. I didn't say the problem was impossible to solve, just that it is an important problem that needs solving. Azul solved the technology issues (or so they claim) but not the economics of it, and they solved it for a language that isn't a good fit for in-memory computing in the first place (to put it politely).
If I have to write software today that keeps a lot of data in-memory and requires reasonable latency (and I do) my only realistic option is C++.
I know all the drawbacks of naive reference counting. C++ shared pointers are a horrible kludge. Fortunately they are only needed in very few places, thanks to RAII and unique_ptr. The problem is that C++ has tons of other issues that will never be solved (antiquated modularity, header files, crazy compile times, excessive complexity and generally lots of baggage from the past).
If I have to write software today that keeps a lot of data in-memory and requires reasonable latency (and I do) my only realistic option is C++.
I don't necessarily have a fundamental disagreement here, but I offer two caveats (one of which you may consider disagreement at a certain level).
One is that a lot of the discussion in this thread is not about what is possible today, but about where language implementations can realistically go.
The second is that there are language implementations that do allow you to keep lots of data in memory and still give you low latency; the catch is that most of them are functional programming languages and do not tie into the ecosystems of C++, Java, etc. which limits their applicability. But consider for example, that Jane Street Capital, which has pretty significant requirements for latency, is using OCaml. This is in part because OCaml has an incremental garbage collector for major collections (in addition to generational garbage collection). As I said, it's not rocket science.
The same goes for Erlang, which uses a model of thread-local heaps. Erlang heaps tend to be small (lots of threads [1] with little data each, even though the total memory footprint may be in the gigabytes), so Erlang can use a straightforward garbage collector that can run concurrently with other threads, does a complete collection fast (because the entire heap fits inside the L3 or even L2 cache) or can avoid collection entirely (if you specify the heap size explicitly with spawn_opt and no collection is needed before thread termination). As a result, Erlang can easily satisfy the low latency requirements for telecommunications (where its being used primarily).
Functional programming languages just had an unavoidable need for garbage collection for half a century now and so implementations of functional languages have seen a lot more GC tuning than imperative languages. Thus, you do at least in theory have other options, but I expect "realistic" in your case also implies being able to tap into certain existing software ecosystems.
Let me finally note that a big problem has arguably been too much focus on the JVM; not that there's anything wrong with supporting the JVM, but it has too often come at the cost of alternative execution models. JIT compilation, lack of value types, a very general threading model, loss of static type information, etc. all can very much get in the way of tuning automatic memory management. Luckily, some of the newer emergent languages target alternative backends, in part to avoid these problems.
[1] Technically, Erlang uses the term "process" in lieu of "thread"; I'm using "thread" to avoid confusion with OS-level processes.
Modern GCs use multiple threads to collect garbage. Also, it is possible to design a GC that does not pause mutator (e.g. C4) or does most of the things concurrently (CMS) or does incremental sweeps (G1 and other evacuating collectors), or uses separate isolated heaps for each thread (Erlang GC).
There are some non-defered reference counting that can offer pretty good performance(within 19% of defered reference counting), while still offering determinism[1].
Yes, but they have the same problem that they require compiler smarts; i.e., they require more than a template to implement shared_ptr<T>. When you're doing that, you may just as well implement something more sophisticated for automated memory management.
The attraction of RAII is that it's a pretty simple mechanism that doesn't require compiler intervention outside of basic, already existing optimizations such as inlining.
- RAII-style freeing means a lot of reference counting. Reference counting and multiple threads don't mix well. You can circumvent that by having multiple separate heaps, but that's not always a practical solution.
- Reference counting is slow; if you want GC-like speed and RC-like pause time guarantees, you're probably better off with deferred reference counting, but that's not easy to engineer into C++; it's essentially a form of garbage collection.
- Lack of fully automatic memory management may affect modularity adversely; see Jones/Lins for the gory details. In brief, wholly or partly manual memory management creates leaky abstractions (this is if you eschew RC for some of your code/types for speed reasons or because you need to deal with cyclic references).
- It is not difficult to have scoped deallocation for resources (scope statement in D, using statement in C#, etc., destructor pragma in Nimrod). Resource lifetime, in any event, does not always coincide with the lifetime of a variable, so this is an incomplete solution. Using GC does not mean not using RAII where its downsides do not matter.