“[An IPv4 address space] black market already exists, albeit on a small scale…[currently] IPv4 addresses are still relatively easy to get…[some believe] that regional registries such as ARIN should head off a potentially deleterious black market by creating a “white market” with established rules for trading IPv4 addresses at market-established costs…But the opportunity to cleanly switch from IPv4 to IPv6 passed many years ago. The current transition strategy, called “dual stack,” requires businesses to remain connected to both IPv4 and IPv6 networks until most of the Internet gets to “the other side” — a process expected to take at least five years.”

“Frosted windows may never be private again”

“a fundamental limit to the level of privacy that is possible when social networks are mined for recommendations”

“The 605-page [NSA IAD] PDF document reads like a listing of the pros and cons for a huge array of defensive and counterintelligence approaches and technologies that an entity might adopt in defending its networks…[one] section delves into the challenges of attributing the true origin(s) of a computer network attack”

“It’s safe to say that when someone pays that much for a bug, they’re not going to tell the vendor to patch it.”

“Regulation is not the primary driver for new technology, new investment, or new training; the threats are”

Protecting Europe against large-scale cyber attacks

Would you have spotted this ATM skimmer?

DoE and power grid security

The Washington Post gave [former DIRNSA and DNI] McConnell free space to declare that we are losing some sort of cyberwar…But that’s not warfare. That’s espionage…Those enamored with the idea of “cyberwar” aren’t dissuaded by fact-checking…[if the DoS attack on Estonia] was cyberwar, it’s pretty clear the net will be just fine. In fact, none of [the commonly cited examples] demonstrate the existence of a cyberwar, let alone that we are losing it. But this battle isn’t about truth. It’s about power…

the problem with developing cyberweapons…is that you need to know where to point them…The military needs targets…Never shy of extending its power, the military industrial complex wants to turn the internet into yet another venue for an arms race. And it’s waging a psychological warfare campaign on the American people to make that so. The military industrial complex is backed by sensationalism, and a gullible and pageview-hungry media…

There is no cyberwar and we are not losing it. The only war going on is one for the soul of the internet. But if…self-interested exaggerators dominate our nation’s discourse about online security, we will lose that war — and the open internet will be its biggest casualty.

On the opposite end of the nuance spectrum: more than 41% of the zeros of the zeta function are on the critical line.

The question for now is, once you’ve got a finite Markov process, what do you do with it? There are some obvious things. For example, you could apply a Chebyshev-type inequality to detect when the traffic parameters change or the underlying assumptions break down (which, if the model is halfway decent, by definition indicates something interesting is going on–even if it’s not malicious). This idea has been around in network security at least since Denning’s 1986-7 intrusion detection article, though, so it’s not likely to bear any more fruit (assuming it ever did). A better idea is to construct and exploit martingales. One way to do this to advantage starting with an inhomogeneous Poisson process (or in principle, at least, more general one-dimensional point processes) was outlined here and here.

Probably the most well-known general technique for constructing martingales from Markov processes is the Dynkin formula. Although we don’t use this formula at present (after having done a lot of tinkering and evaluation), a more general result similar to it will help us introduce the Girsanov theorem for finite Markov processes and thereby one of the tools we’ve developed for detecting changes in network traffic patterns.

The sketch below of a fairly general version of this formula for finite processes is adapted from a preprint of Ford (see Rogers and Williams IV.20 for a more sophisticated treatment).

Consider a time-inhomogeneous Markov process on a finite state space. Let denote the generator, and let denote the corresponding transition kernel, i.e. where the Markov propagator is

and indicates the formal adjoint or reverse time-ordering operator. Thus, e.g., an initial distribution is propagated as (NB. Kleinrock‘s queueing theory book omits the time-ordering, which is a no-no.)

Let be bounded and such that the map is Write and Now

and the Markov property gives that

The notation just indicates the history of the process (i.e., its natural filtration) at time The transition kernel satisfies a generalization of the time-homogeneous formula

so the RHS of the previous equation is times

plus a term that vanishes in the limit of vanishing mesh. The fact that the row sums of a generator are identically zero has been used to simplify the result.

Summing over and taking the limit as the mesh of the the partition goes to zero shows that

That is,

is a local martingale, or if is well behaved, a martingale.

This can be generalized (see Rogers and Williams IV.21 and note that the extension to inhomogeneous processes is trivial): if is an inhomogeneous Markov process on a finite state space and is such that is locally bounded and previsible and for all then given by

is a local martingale. Conversely, any local martingale null at 0 can be represented in this form for some satisfying the conditions above (except possibly local boundedness).

To reiterate, this result will be used to help introduce the Girsanov theorem for finite Markov processes in a future post, and later on we’ll also show how Girsanov can be used to arrive at a genuinely simple, scalable likelihood ratio test for identifying changes in network traffic patterns.

DDoS stats from Arbor

Adobe gets attacked

“Complex software-hardware systems are not proved correct [in 1976], they are not proved correct today, and will not be proved correct in the future. If I am wrong why do we need cyber-security? Why do systems get attacked and broken into every day? Why is security a multi-billion dollar a year industry? I think the answer is obvious.”

National Cyber Range awards

Entropy, entropy, entropy…

“We spend billions on security, yet we are not any more secure. We have lots of regulations, but that has created a low bar mentality”

Single-qubit experimental quantum computation for the Jones polynomial

Colloidal metamaterial…er…“liquid invisibility cloak”

Visualizing Abdulmutallab. This is supposed to make some sense if you look at it long enough, apparently.

Geolocation hack

IPv4 lives on…for now

Suricata IDS in beta. Another open-source IDS is a good thing. (But open-source network monitoring will be even better!)

The best defense is a good offense

Switchable DNA nanostructures

Hijacking NetBIOS

Eavesdropping on quantum crypto?

Survey of key exchange security deriving from the Second Law

An approach to subexponential factoring

The use of ideas of Information Theory for studying “language” and intelligence in ants

In this post I’m going to talk about a way in which one of the simplest models from statistical physics might inform group dynamics in birds (and probably even people in similar situations). As far as I know, the experiment hasn’t been done–the closest work to it seems to be on flocking (though I’ll give $.50 and a Sprite to the first person to point out a direct reference to this sort of thing). I’ve been kicking it around for years and I think that at varying scopes and levels of complexity, it might constitute anything from a really good high school science fair project to a PhD dissertation. In fact I may decide to run with this idea myself some day, and I hope that anyone else out there who wants to do the same will let me know.

The basic idea is simple. But first let me show you a couple of pictures.

Notice how the tree in the picture above looks? There doesn’t seem to be any wind. But I bet that either the birds flocked to the wire together or there was at least a breeze when the picture below was taken:

Because the birds are on wires, they can face in essentially one of two directions. In the first picture it looks very close to a 60%-40% split, with most of the roughly 60 birds facing left. In the second picture, 14 birds are facing right and only one is facing left.

Now let me show you an equation:

If you are a physicist you already know that this is the Hamiltonian for the spin-1/2 Ising model with an applied field, but I will explain this briefly. The Hamiltonian is really just a fancy word for energy. It is the energy of a model (notionally magnetic) system in which spins that occupy sites that are (typically) on a lattice (e.g., a one-dimensional lattice of equally spaced points) take the values and can be taken as caricatures of dipoles. The notation indicates that the first sum is taken over nearest neighbors in the lattice: the spins interact, but only with their neighbors, and the strength of this interaction is reflected in the exchange energy The strength of the spins’ interaction with an applied (again notionally magnetic) field is governed by the field strength This is the archetype of spin models in statistical physics, and it won’t serve much for me to reproduce a discussion that can be found many other places (you may like to refer to Goldenfeld’s Lectures on Phase Transitions and the Renormalization Group, which also covers the the renormalization group method that inspires the data reduction techniques used in our software). Suffice it to say that these sorts of models comprise a vast field of study and already have an enormous number of applications in lots of different areas.

Now let me talk about what the pictures and the model have in common. The (local or global) average spin is called the magnetization. Ignoring an arbitrary sign, in the first picture the magnetization is roughly 0.2, and in the second it’s about 0.87. The 1D spin-1/2 Ising model is famous for exhibiting a simple phase transition in magnetization: indeed, the expected value of the magnetization for in the thermodynamic limit is shown in every introductory statistical physics course worth the name to be

where is the inverse temperature (in natural units). As ever, a picture is worth a thousand words:

For and it’s easy to see that But if and , then taking the subsequent limit yields a magnetization of At zero temperature the model becomes completely magnetized–i.e., totally ordered. (Finite-temperature phase transitions in magnetization in the real world are of paramount importance for superconductivity.)

And at long last, here’s the point. I am willing to bet ($.50 and a Sprite, as usual) that the arrangement of birds on wires can be well described by a simple spin model, and probably the spin-1/2 Ising model provided that the spacing between birds isn’t too wide. I expect that the same model with varying parameters works for many–or even most or all–species in some regime, which is a bet on a particularly strong kind of universality. Neglecting spacing between birds, I expect the effective exchange strength to depend on the species of bird, and the effective applied field to depend on the wind speed and angle, and possibly the sun’s relative location (and probably a transient to model the effects of arriving on the wire in a flock). I don’t have any firm suspicions on what might govern an effective temperature here, but I wouldn’t be surprised to see something that could be well described by Kawasaki or Glauber dynamics for spin flips: that is, I reckon that–as usual–it’s necessary to take timescales into account in order to unambiguously assign a formal or effective temperature (if the birds effectively stay still, then dynamics aren’t relevant and the temperature should be regarded as being already accounted for in the exchange and field parameters). I used to think about doing this kind of experiment using tagged photographs or their ilk near windsocks or something similar, but I can’t see how to get any decent results that way without more effort than a direct experiment. I think it probably ought to be done (at least initially) in a controlled environment.

Anyways, there it is. The experiment always wins, but I have a hunch how it would turn out.

UPDATE 30 Jan 2010: Somebody had another interesting idea involving birds on wires.

]]> http://blog.eqnets.com/2009/11/30/birds-on-a-wire-and-the-ising-model/feed/ 6 eqnets birds1 birds2 magnetization