We’ve just posted a paper titled “Effective statistical physics of Anosov systems” that details the physical relevance of the techniques we’ve used to characterize network traffic. The idea is that there appears to be a unique well-defined effective temperature (and energy spectrum) for physical systems that are typical under the so-called chaotic hypothesis. We’ve demonstrated how statistical physics can be used to detect malicious or otherwise anomalous network traffic in another whitepaper also available on the arxiv through our downloads page. The current paper completes the circle and presents evidence indicating that the same ideas can be fruitfully applied to nonequilbrium steady states.
Can you describe in greater detail how exactly anosov flow is used to model computer networks? Your other paper explains the idea of defining the idea of a temperature and ebuilibrium for computer network but how is this better than the bose gas model for example?
Thanks for your question. I hope that based on your IP address you may be interested in a fairly technical and discursive answer.
Anyway, Anosov systems aren’t used at all. The idea of studying them was to bolster the physical legitimacy of and elaborate the theory of the effective temperature more generally. Equilibrium Networks owes a lot to theoretical work done with the practical problem of network defense in mind. And it is the topic of my dissertation work in physics, so there’s that.
When we were characterizing (not modeling) networks using temperature etc, the Bose gas framework was always used. However a couple of months ago we terminated a NSA license to use effective temperatures and energies to characterize network traffic. The reason is simple: as promising as the idea appears (and it did work quite well on the tests we were able to perform), there are much simpler ways to close the loop with actionable information, and it didn’t make sense for us to keep developing or marketing the capability.
In more detail: having (e.g.) a work rate or power component spike doesn’t give you anything more than a general alert. For practical purposes, it’s totally infeasible to have a continuous buffer of packet information (too many flops, too much storage space for numerical data structures, etc), and instead it’s necessary to collect information in cycles (say at a nominal 1 Hz). That kills the time resolution of any time series and when all you have is a “something happened last second” it’s not much help. If we could keep perfect time resolution, we could use the timestamp as a hash to pull up the packet triggering the alert and get to flows or sessions without too much more effort…but we can’t.
This sort of “alert to information” problem was addressed in a general context by [Filho, F. J. S. Unsupervised Diagnosis of Network Traffic Anomalies. PhD thesis, Université Paris VI (2010)].
That said, one could in principle try something that is more manifestly physically interesting as follows. Let transmission or reception on a network correspond to creation/annihilation operators on a Fock space of directed edges. In a continuum limit, we’d expect to get a field theory. Its parameters are the energies of single- particle eigenstates, which could be specified through partial observation of network activity (e.g., with Bayesian methods) and the effective temperature framework. This would suggest a stochastic analogue of the renormalization group and probably require considerable work.
BTW, have you actually tried out our system? I’m always interested to hear about it.
Thanks again and sincerely
Steve Huntsman
Thanks for the explanation and I have not actually tried your product yet. Can you think of anyway to model a network as anosov system? I’ve seen this idea in the past briefly but I can’t see any such method. Are there any authoritative papers on the fock space approach?
I can’t think of such a way. As far as I know the Fock space idea has not been considered elsewhere, though it’s possible that I’m cryptomnesiac. AFAIK/IMO the Fock space idea would represent a significant research program, and in fact I pitched it (without success) to ONR for that reason.
Also do you mind explaining the idea of internal states in your bose gas approach, you mentioned it allows you to implement sophisticated internal states such as “ip addresses inside the network and present in observed traffic between 1 and 10 times during the last 5 seconds”
The internal states are attributes common to coarse-grained notions of source and destination. At the IP level, this means specified IPs or those occurring with some specific frequency. Same at the port level. At the “flow” level, it amounts to the “attached” and “unattached” attributes described in our whitepapers and poster on the downloads page. Look for the decision trees in these documents for more information.
Thanks again this is a bit unrelated but how did you leverage snort for your application.
Our basic sensor is a Snort preprocessor. It is actually quite fast as- is (i.e., without a modified Linux kernel or specialized data capture card) but there are some tradeoffs that have to be made (e.g., only sampling traffic for storage, though we do this fairly rationally).
To overcome this, we designed a fuser that lets you run Snort instances in parallel. This is an experimental capability only though.
Frankly I am surprised that more people outside Sourcefire haven’t built Snort preprocessors (there are only very few, and fewer serious tools). It provides a good substrate.
We had participated in feasibility studies for sensors operating with full storage and maximum throughput at 10 to 40/100 Gbps. Some promising prototypes were built by another firm but unfortunately the code is not available.
Thanks again. I actually found your website through your answer on the heat kernel proof of the atiyah singer on MOF a while back and this lead to a whole slew of interesting applications and ideas found in your blog. You have a very cool job for sure.
I’ll works this look throughout to 2 types of people: current Zune owners who are in view of an upgrade, and people vexing to write up one’s aptitude between a Zune and an iPod. (There are other players gain account unserviceable there, like the Sony Walkman X, but I trust this gives you adequately info to compel passable an conversant with outcome of the Zune vs players other than the iPod heterogeneity role as well.)