This article, as written by me, appeared in The Hindu on January 10, 2012.
--
After a successful three-year run that saw the discovery of a Higgs-boson-like particle in early 2012, the Large Hadron Collider (LHC) at CERN, near Geneva, Switzerland, will shut down for 18 months for maintenance and upgrades.
This is the first of three long shutdowns, scheduled for 2013, 2017, and 2022. Physicists and engineers will use these breaks to ramp up one of the most sophisticated experiments in history even further.
According to Mirko Pojer, Engineer In-charge, LHC-operations, most of these changes were planned in 2011. They will largely concern fixing known glitches on the ATLAS and CMS particle-detectors. The collider will receive upgrades to increase its collision energy and frequency.
Presently, the LHC smashes two beams, each composed of precisely spaced bunches of protons, at 3.5-4 tera-electron-volts (TeV) per beam.
By 2015, the beam energy will be pushed up to 6.5-7 TeV per beam. Moreover, the bunches which were smashed at intervals of 50 nanoseconds will do so at 25 nanoseconds.
After upgrades, “in terms of performance, the LHC will deliver twice the luminosity,” Dr. Pojer noted in an email to this Correspondent, with reference to the integrated luminosity. Precisely, it is the number of collisions that the LHC can deliver per unit area which the detectors can track.
The instantaneous luminosity, which is the luminosity per second, will be increased to 1x1034 per centimetre-squared per second, ten-times greater than before, and well on its way to peaking at 7.73x1034 per centimetre-squared per second by 2022.
As Steve Myers, CERN’s Director for Accelerators and Technology, announced in December 2012, “More intense beams mean more collisions and a better chance of observing rare phenomena.” One such phenomenon is the appearance of a Higgs-boson-like particle.
The CMS experiment, one of the detectors on the LHC-ring, will receive some new pixel sensors, a technology responsible for tracking the paths of colliding particles. To make use of the impending new luminosity-regime, an extra layer of these advanced sensors will be inserted around a smaller beam pipe.
If results from it are successful, CMS will receive the full unit in late-2016.
In the ATLAS experiment, unlike with CMS which was built with greater luminosities in mind, pixel sensors are foreseen to wear out within one year after upgrades. As an intermediate solution, a new layer of sensors called the B-layer will be inserted within the detector for until 2018.
Because of the risk of radiation damage due to more numerous collisions, specific neutron shields will be fit, according to Phil Allport, ATLAS Upgrade Coordinator.
Both ATLAS and CMS will also receive evaporative cooling systems and new superconducting cables to accommodate the higher performance that will be expected of them in 2015. The other experiments, LHCb and ALICE, will also undergo inspections and upgrades to cope with higher luminosity.
An improved failsafe system will be installed and the existing one upgraded to prevent accidents such as the one in 2008.
Then, an electrical failure damaged 29 magnets and leaked six tonnes of liquid helium into the tunnel, precipitating an eight-month shutdown.
Generally, as Martin Gastal, CMS Experimental Area Manager, explained via email, “All sub-systems will take the opportunity of this shutdown to replace failing parts and increase performance when possible.”
All these changes have been optimised to fulfil the LHC’s future agenda. This includes studying the properties of the newly discovered particle, and looking for signs of new theories of physics like supersymmetry and higher dimensions.
(Special thanks to Achintya Rao, CMS Experiment.)
Wednesday, 9 January 2013
Tuesday, 1 January 2013
The case of the red-haired kids
This blog post first appeared, as written by me, on The Copernican science blog on December 30, 2012.
--
Seriously, shame on me for not noticing the release of a product named Correlate until December 2012. Correlate by Google was released in May last year and is a tool to see how two different search trends have panned out over a period of time. But instead of letting you pick out searches and compare them, Correlate saves a bit of time by letting you choose one trend and then automatically picks out trends similar to the one you’ve your eye on.
For instance, I used the “Draw” option and drew a straight, gently climbing line from September 19, 2004, to July 24, 2011 (both randomly selected). Next, I chose “India” as the source of search queries for this line to be compared with, and hit “Correlate”. Voila! Google threw up 10 search trends that varied over time just as my line had.

Since I’ve picked only India, the space from which the queries originate remains fixed, making this a temporal trend - a time-based one. If I’d fixed the time - like a particular day, something short enough to not produce strong variations - then it’d have been a spatial trend, something plottable on a map.
Now, there were a lot of numbers on the results page. The 10 trends displayed in fact were ranked according to a particular number “r” displayed against them. The highest ranked result, “free english songs”, had r = 0.7962. The lowest ranked result, “to 3gp converter”, had r = 0.7653.

And as I moused over the chart itself, I saw two numbers, one each against the two trends being tracked. For example, on March 1, 2009, the “Drawn Series” line had a number +0.701, and the “free english songs” line had a number -0.008, against it.

What do these numbers mean?
This is what I want to really discuss because they have strong implications on how lay people interpret data that appears in the context of some scientific text, like a published paper. Each of these numbers is associated with a particular behaviour of some trend at a specific point. So, instead of looking at it as numbers and shapes on a piece of paper, look at it for what it represents and you’ll see so many possibilities coming to life.
The numbers against the trends, +0.701 for “Drawn Series” (my line) and -0.008 for “free english songs” in March ‘09, are the deviations. The deviation is a lovely metric because it sort of presents the local picture in comparison to the global picture, and this perspective is made possible by the simple technique used to evaluate it.
Consider my line. Each of the points on the line has a certain value. Use this information to find their average value. Now, the deviation is how much a point’s value is away from the average value.
It’s like if 11 red-haired kids were made to stand in a line ordered according to the redness of their hair. If the “average” colour around was a perfect orange, then the kid with the “reddest” hair and the kid with the palest-red hair will be the most deviating. Kids with some semblance of orange in their hair-colour will be progressively less deviating until they’re past the perfect “orangeness”, and the kid with perfectly-orange hair will completely non-deviating.
So, on August 23, 2009, “Drawn Series” was higher than its average value by 0.701 and “free english songs” was lower than its average value by 0.008. Now, if you’re wondering what the units are to measure these numbers: Deviations are dimensionless fractions - which means they’re just numbers whose highness or lowness are indications of intensity.
And what’re they fractions of? The value being measured along the trend being tracked.
Now, enter standard deviation. Remember how you found the average value of a point on my line? Well, the standard deviation is the average value among all deviations. It’s like saying the children fitting a particular demographic are, for instance, 25 per cent smarter on average than other normal kids: the standard deviation is 25 per cent and the individual deviations are similar percentages of the “smartness” being measured.
So, right now, if you took the bigger picture, you’d see the chart, the standard deviation (the individual deviations if you chose to mouse-over), the average, and that number “r”. The average will indicate the characteristic behaviour of the trend - let’s call it “orange” - the standard deviation will indicate how far off on average a point’s behaviour will be deviating in comparison to “orange” - say, “barely orange”, “bloody”, etc. - and the individual deviations will show how “orange” each point really is.
At this point I must mention that I conveniently oversimplified the example of the red-haired kids to avoid a specific problem. This problem has been quite single-handedly responsible for the news-media wrongly interpreting results from the LHC/CERN on the Higgs search.
In the case of the kids, we assumed that, going down the line, each kid’s hair would get progressively darker. What I left out was how much darker the hair would get with each step.
Let’s look at two different scenarios.
Scenario 1: The hair gets darker by a fixed amount each step.
Let’s say the first kid’s got hair that’s 1 units of orange, the fifth kid’s got 5 units, and the 11th kid’s got 11 units. This way, the average “amount of orange” in the lineup is going to be 6 units. The deviation on either side of kid #6 is going to increase/decrease in steps of 1. In fact, from the first to the last, it’s going to be 5, 4, 3, 2, 1, 0, 1, 2, 3, 4, and 5. Straight down and then straight up.

Scenario 2: The hair gets darker slowly and then rapidly, also from 1 to 11 units.
In this case, the average is not going to be 6 units. Let’s say the “orangeness” this time is 1, 1.5, 2, 2.5, 3, 3.5, 4, 5.5, 7.5, 9.75, and 11 per kid, which brings the average to ~4.6591 units. In turn, the deviations are 3.6591, 3.1591, 2.6591, 2, 1591, 1.6591, 1.1591, 0.6591, 0.8409, 2.8409, 5.0909, and 6.3409. In other words, slowly down and then quickly more up.

In the second scenario, we saw how the average got shifted to the left. This is because there were more less-orange kids than more-orange ones. What’s more important is that it didn’t matter if the kids on the right had more more-orange hair than before. That they were fewer in number shifted the weight of the argument away from them!
In much the same way, looking for the Higgs boson from a chart that shows different peaks (number of signature decay events) at different points (energy levels), with taller but fewer peaks to one side and shorter but many more peaks to the other, can be confusing. While more decays could’ve occurred at discrete energy levels, the Higgs boson is more likely (note: not definitely) to be found within the energy-level where decays occur more frequently (in the chart below, decays are seen to occur more frequently at 118-126 GeV/c2 than at 128-138 GeV/c2 or 110-117 GeV/c2).
[caption id="attachment_24766" align="aligncenter" width="750"]
Idea from Prof. Matt Strassler's blog[/caption]
If there’s a tall peak where a Higgs isn’t likely to occur, then that’s an outlier, a weirdo who doesn’t fit into the data. It’s probably called an outlier because its deviation from the average could be well outside the permissible deviation from the average.
This also means it’s necessary to pick the average from the right area to identify the right outliers. In the case of the Higgs, if its associated energy-level (mass) is calculated as being an average of all the energy levels at which a decay occurs, then freak occurrences and statistical noise are going to interfere with the calculation. But knowing that some masses of the particle have been eliminated, we can constrain the data to between two energy levels, and then go after the average.
So, when an uninformed journalist looks at the data, the taller peaks can catch the eye, even run away with the ball. But look out for the more closely occurring bunches - that’s where all the action is!
If you notice, you’ll also see that there are no events at some energy levels. This is where you should remember that uncertainty cuts both ways. When you’re looking at a peak and thinking “This can’t be it; there’s some frequency of decays to the bottom, too”, you’re acknowledging some uncertainty in your perspective. Why not acknowledge some uncertainty when you’re noticing absent data, too?
While there’s a peak at 126 GeV/c2, the Higgs weighs between 124-125 GeV/c2. We know this now, so when we look at the chart, we know we were right in having been uncertain about the mass of the Higgs being 126 GeV/c2. Similarly, why not say “There’s no decays at 113 GeV/c2, but let me be uncertain and say there could’ve been a decay there that’s escaped this measurement”?
Maybe this idea’s better illustrated with this chart.

There’s a noticeable gap between 123 and 125 GeV/c2. Just looking at this chart and you’re going to think that with peaks on either side of this valley, the Higgs isn’t going to be here… but that’s just where it is! So, make sure you address uncertainty when you’re determining presences as well as absences.
So, now, we’re finally ready to address “r”, the Pearson covariance coefficient. It’s got a formula, and I think you should see it. It’s pretty neat.

(TeX: r\quad =\quad \frac { { \Sigma }_{ i=1 }^{ n }({ X }_{ i }\quad -\quad \overset { \_ }{ X } )({ Y }_{ i }\quad -\quad \overset { \_ }{ Y } ) }{ \sqrt { { \Sigma }_{ i=1 }^{ n }{ ({ X }_{ i }\quad -\quad \overset { \_ }{ X } ) }^{ 2 } } \sqrt { { \Sigma }_{ i=1 }^{ n }{ (Y_{ i }\quad -\quad \overset { \_ }{ Y } ) }^{ 2 } } })
The equation says "Let's see what your Pearson covariance, "r", is by seeing how much all of your variations are deviant keeping in mind both your standard deviations."
The numerator is what’s called the covariance, and the denominator is basically the product of the standard deviations. X-bar, which is X with a bar atop, is the average value of X - my line - and the same goes for Y-bar, corresponding to Y - “mobile games”. Individual points on the lines are denoted with the subscript “i”, so the points would be X1, X2, X3, ..., and Y1, Y2, Y3, …”n” in the formula is the size of the sample - the number of days over which we’re comparing the two trends.
The Pearson covariance coefficient is not called the Pearson deviation coefficient, etc., because it normalises the graph’s covariance. Simply put, covariance is a measure of how much the two trends vary together. It can have a minimum value of 0, which would mean one trend’s variation has nothing to do with the other’s, and a maximum value of 1, which would mean one trend’s variation is inescapably tied with the variation of the other’s. Similarly, if the covariance is positive, it means that if one trend climbs, the other would climb, too. If the covariance is negative, then one trend’s climbing would mean the other’s descending (In the chart below, between Oct ’09 and Jan ’10, there’s a dip: even during the dive-down, the blue line is on an increasing note – here, the local covariance will be negative).

Apart from being a conveniently defined number, covariance also records a trend’s linearity. In statistics, linearity is a notion that stands by its name: like a straight line, the rise or fall of a trend is uniform. If you divided up the line into thousands of tiny bits and called each one on the right the “cause” and the one on the left the “effect”, then you’d see that linearity means each effect for each cause is either an increase or a decrease by the same amount.
Just like that, if the covariance is a lower positive number, it means one trend’s growth is also the other trend’s growth, and in equal measure. If the covariance is a larger positive number, you’d have something like the butterfly effect: one trend moves up by an inch, the other shoots up by a mile. This you’ll notice is a break from linearity. So if you plotted the covariance at each point in a chart as a chart by itself, one look will tell you how the relationship between the two trends varies over time (or space).
--
Seriously, shame on me for not noticing the release of a product named Correlate until December 2012. Correlate by Google was released in May last year and is a tool to see how two different search trends have panned out over a period of time. But instead of letting you pick out searches and compare them, Correlate saves a bit of time by letting you choose one trend and then automatically picks out trends similar to the one you’ve your eye on.
For instance, I used the “Draw” option and drew a straight, gently climbing line from September 19, 2004, to July 24, 2011 (both randomly selected). Next, I chose “India” as the source of search queries for this line to be compared with, and hit “Correlate”. Voila! Google threw up 10 search trends that varied over time just as my line had.
Since I’ve picked only India, the space from which the queries originate remains fixed, making this a temporal trend - a time-based one. If I’d fixed the time - like a particular day, something short enough to not produce strong variations - then it’d have been a spatial trend, something plottable on a map.
Now, there were a lot of numbers on the results page. The 10 trends displayed in fact were ranked according to a particular number “r” displayed against them. The highest ranked result, “free english songs”, had r = 0.7962. The lowest ranked result, “to 3gp converter”, had r = 0.7653.
And as I moused over the chart itself, I saw two numbers, one each against the two trends being tracked. For example, on March 1, 2009, the “Drawn Series” line had a number +0.701, and the “free english songs” line had a number -0.008, against it.
What do these numbers mean?
This is what I want to really discuss because they have strong implications on how lay people interpret data that appears in the context of some scientific text, like a published paper. Each of these numbers is associated with a particular behaviour of some trend at a specific point. So, instead of looking at it as numbers and shapes on a piece of paper, look at it for what it represents and you’ll see so many possibilities coming to life.
The numbers against the trends, +0.701 for “Drawn Series” (my line) and -0.008 for “free english songs” in March ‘09, are the deviations. The deviation is a lovely metric because it sort of presents the local picture in comparison to the global picture, and this perspective is made possible by the simple technique used to evaluate it.
Consider my line. Each of the points on the line has a certain value. Use this information to find their average value. Now, the deviation is how much a point’s value is away from the average value.
It’s like if 11 red-haired kids were made to stand in a line ordered according to the redness of their hair. If the “average” colour around was a perfect orange, then the kid with the “reddest” hair and the kid with the palest-red hair will be the most deviating. Kids with some semblance of orange in their hair-colour will be progressively less deviating until they’re past the perfect “orangeness”, and the kid with perfectly-orange hair will completely non-deviating.
So, on August 23, 2009, “Drawn Series” was higher than its average value by 0.701 and “free english songs” was lower than its average value by 0.008. Now, if you’re wondering what the units are to measure these numbers: Deviations are dimensionless fractions - which means they’re just numbers whose highness or lowness are indications of intensity.
And what’re they fractions of? The value being measured along the trend being tracked.
Now, enter standard deviation. Remember how you found the average value of a point on my line? Well, the standard deviation is the average value among all deviations. It’s like saying the children fitting a particular demographic are, for instance, 25 per cent smarter on average than other normal kids: the standard deviation is 25 per cent and the individual deviations are similar percentages of the “smartness” being measured.
So, right now, if you took the bigger picture, you’d see the chart, the standard deviation (the individual deviations if you chose to mouse-over), the average, and that number “r”. The average will indicate the characteristic behaviour of the trend - let’s call it “orange” - the standard deviation will indicate how far off on average a point’s behaviour will be deviating in comparison to “orange” - say, “barely orange”, “bloody”, etc. - and the individual deviations will show how “orange” each point really is.
At this point I must mention that I conveniently oversimplified the example of the red-haired kids to avoid a specific problem. This problem has been quite single-handedly responsible for the news-media wrongly interpreting results from the LHC/CERN on the Higgs search.
In the case of the kids, we assumed that, going down the line, each kid’s hair would get progressively darker. What I left out was how much darker the hair would get with each step.
Let’s look at two different scenarios.
Scenario 1: The hair gets darker by a fixed amount each step.
Let’s say the first kid’s got hair that’s 1 units of orange, the fifth kid’s got 5 units, and the 11th kid’s got 11 units. This way, the average “amount of orange” in the lineup is going to be 6 units. The deviation on either side of kid #6 is going to increase/decrease in steps of 1. In fact, from the first to the last, it’s going to be 5, 4, 3, 2, 1, 0, 1, 2, 3, 4, and 5. Straight down and then straight up.
Scenario 2: The hair gets darker slowly and then rapidly, also from 1 to 11 units.
In this case, the average is not going to be 6 units. Let’s say the “orangeness” this time is 1, 1.5, 2, 2.5, 3, 3.5, 4, 5.5, 7.5, 9.75, and 11 per kid, which brings the average to ~4.6591 units. In turn, the deviations are 3.6591, 3.1591, 2.6591, 2, 1591, 1.6591, 1.1591, 0.6591, 0.8409, 2.8409, 5.0909, and 6.3409. In other words, slowly down and then quickly more up.
In the second scenario, we saw how the average got shifted to the left. This is because there were more less-orange kids than more-orange ones. What’s more important is that it didn’t matter if the kids on the right had more more-orange hair than before. That they were fewer in number shifted the weight of the argument away from them!
In much the same way, looking for the Higgs boson from a chart that shows different peaks (number of signature decay events) at different points (energy levels), with taller but fewer peaks to one side and shorter but many more peaks to the other, can be confusing. While more decays could’ve occurred at discrete energy levels, the Higgs boson is more likely (note: not definitely) to be found within the energy-level where decays occur more frequently (in the chart below, decays are seen to occur more frequently at 118-126 GeV/c2 than at 128-138 GeV/c2 or 110-117 GeV/c2).
[caption id="attachment_24766" align="aligncenter" width="750"]
If there’s a tall peak where a Higgs isn’t likely to occur, then that’s an outlier, a weirdo who doesn’t fit into the data. It’s probably called an outlier because its deviation from the average could be well outside the permissible deviation from the average.
This also means it’s necessary to pick the average from the right area to identify the right outliers. In the case of the Higgs, if its associated energy-level (mass) is calculated as being an average of all the energy levels at which a decay occurs, then freak occurrences and statistical noise are going to interfere with the calculation. But knowing that some masses of the particle have been eliminated, we can constrain the data to between two energy levels, and then go after the average.
So, when an uninformed journalist looks at the data, the taller peaks can catch the eye, even run away with the ball. But look out for the more closely occurring bunches - that’s where all the action is!
If you notice, you’ll also see that there are no events at some energy levels. This is where you should remember that uncertainty cuts both ways. When you’re looking at a peak and thinking “This can’t be it; there’s some frequency of decays to the bottom, too”, you’re acknowledging some uncertainty in your perspective. Why not acknowledge some uncertainty when you’re noticing absent data, too?
While there’s a peak at 126 GeV/c2, the Higgs weighs between 124-125 GeV/c2. We know this now, so when we look at the chart, we know we were right in having been uncertain about the mass of the Higgs being 126 GeV/c2. Similarly, why not say “There’s no decays at 113 GeV/c2, but let me be uncertain and say there could’ve been a decay there that’s escaped this measurement”?
Maybe this idea’s better illustrated with this chart.
There’s a noticeable gap between 123 and 125 GeV/c2. Just looking at this chart and you’re going to think that with peaks on either side of this valley, the Higgs isn’t going to be here… but that’s just where it is! So, make sure you address uncertainty when you’re determining presences as well as absences.
So, now, we’re finally ready to address “r”, the Pearson covariance coefficient. It’s got a formula, and I think you should see it. It’s pretty neat.
(TeX: r\quad =\quad \frac { { \Sigma }_{ i=1 }^{ n }({ X }_{ i }\quad -\quad \overset { \_ }{ X } )({ Y }_{ i }\quad -\quad \overset { \_ }{ Y } ) }{ \sqrt { { \Sigma }_{ i=1 }^{ n }{ ({ X }_{ i }\quad -\quad \overset { \_ }{ X } ) }^{ 2 } } \sqrt { { \Sigma }_{ i=1 }^{ n }{ (Y_{ i }\quad -\quad \overset { \_ }{ Y } ) }^{ 2 } } })
The equation says "Let's see what your Pearson covariance, "r", is by seeing how much all of your variations are deviant keeping in mind both your standard deviations."
The numerator is what’s called the covariance, and the denominator is basically the product of the standard deviations. X-bar, which is X with a bar atop, is the average value of X - my line - and the same goes for Y-bar, corresponding to Y - “mobile games”. Individual points on the lines are denoted with the subscript “i”, so the points would be X1, X2, X3, ..., and Y1, Y2, Y3, …”n” in the formula is the size of the sample - the number of days over which we’re comparing the two trends.
The Pearson covariance coefficient is not called the Pearson deviation coefficient, etc., because it normalises the graph’s covariance. Simply put, covariance is a measure of how much the two trends vary together. It can have a minimum value of 0, which would mean one trend’s variation has nothing to do with the other’s, and a maximum value of 1, which would mean one trend’s variation is inescapably tied with the variation of the other’s. Similarly, if the covariance is positive, it means that if one trend climbs, the other would climb, too. If the covariance is negative, then one trend’s climbing would mean the other’s descending (In the chart below, between Oct ’09 and Jan ’10, there’s a dip: even during the dive-down, the blue line is on an increasing note – here, the local covariance will be negative).
Apart from being a conveniently defined number, covariance also records a trend’s linearity. In statistics, linearity is a notion that stands by its name: like a straight line, the rise or fall of a trend is uniform. If you divided up the line into thousands of tiny bits and called each one on the right the “cause” and the one on the left the “effect”, then you’d see that linearity means each effect for each cause is either an increase or a decrease by the same amount.
Just like that, if the covariance is a lower positive number, it means one trend’s growth is also the other trend’s growth, and in equal measure. If the covariance is a larger positive number, you’d have something like the butterfly effect: one trend moves up by an inch, the other shoots up by a mile. This you’ll notice is a break from linearity. So if you plotted the covariance at each point in a chart as a chart by itself, one look will tell you how the relationship between the two trends varies over time (or space).
Monday, 31 December 2012
2012 in review
The WordPress.com stats helper monkeys prepared a 2012 annual report for this blog.

Here's an excerpt:
Click here to see the complete report.
Here's an excerpt:
4,329 films were submitted to the 2012 Cannes Film Festival. This blog had 22,000 views in 2012. If each view were a film, this blog would power 5 Film Festivals
Click here to see the complete report.
Friday, 21 December 2012
Is there only one road to revolution?
Read this first.
Some of this connects, some of it doesn't. Most of all, I have discovered a fear in me that keeps me from from disagreeing with people like Meena Kandasamy - great orators, no doubt, but what are they really capable of?
The piece speaks of revolution as being the sole goal of an Indian youth's life, that we must spend our lives stirring the muddied water, exposing the mud to light, and separating grime from guts and guts from guts from glory. This is where I disagree. Revolution is not my cause. I don't want to stir the muddied water. I concede that I am afraid that I will fail.
And at this point, Meena Kandasamy would have me believe, I should either crawl back into my liberty-encrusted shell or lay down my life. Why should I when I know I will succeed in keeping aspirations alive? Why should I when, given the freedom to aspire, I can teach others how to go about believing the same? Why should I when I can just pour in more and more clean water and render the mud a minority?
Why is this never an option? Have we reached a head, that it's either a corruption-free world or a bloodied one? India desperately needs a revolution, yes, but not one that welcomes a man liberated after pained struggles to a joyless world.
Some of this connects, some of it doesn't. Most of all, I have discovered a fear in me that keeps me from from disagreeing with people like Meena Kandasamy - great orators, no doubt, but what are they really capable of?
The piece speaks of revolution as being the sole goal of an Indian youth's life, that we must spend our lives stirring the muddied water, exposing the mud to light, and separating grime from guts and guts from guts from glory. This is where I disagree. Revolution is not my cause. I don't want to stir the muddied water. I concede that I am afraid that I will fail.
And at this point, Meena Kandasamy would have me believe, I should either crawl back into my liberty-encrusted shell or lay down my life. Why should I when I know I will succeed in keeping aspirations alive? Why should I when, given the freedom to aspire, I can teach others how to go about believing the same? Why should I when I can just pour in more and more clean water and render the mud a minority?
Why is this never an option? Have we reached a head, that it's either a corruption-free world or a bloodied one? India desperately needs a revolution, yes, but not one that welcomes a man liberated after pained struggles to a joyless world.
Thursday, 20 December 2012
How big is your language?
This blog post first appeared, as written by me, on The Copernican science blog on December 20, 2012.
--
It all starts with Zipf’s law. Ever heard of it? It’s a devious little thing, especially when you apply it to languages.
Zipf’s law states that the chances of finding a word of a language in all the texts written in that language are inversely proportional to the word’s rank in the frequency table. In other words, this means that the chances of finding the most frequent word is twice as much as are chances of finding the second most frequent word, thrice as much as are chances of finding the third most frequent word, and so on.
Unfortunately (only because I like how “Zipf” sounds), the law holds only until about the 1,000th most common word; after this point, a logarithmic plot drawn between frequency and chance stops being linear and starts to curve.
The importance of this break is that if Zipf’s law fails to hold for a large corpus of words, then the language, at some point, must be making some sort of distinction between common and exotic words, and its need for new words must either be increasing or decreasing. This is because, if the need remained constant, then the distinction would be impossible to define except empirically and never conclusively - going against the behaviour of Zipf's law.
Consequently, the chances of finding the 10,000th word won’t be 10,000 times less than the chances of finding the most frequently used word but a value much lesser or much greater.
A language's diktat
Analysing each possibility, i.e., if the chances of finding the 10,000th-most-used word are NOT 10,000 times less than the chances of finding the most-used word but…
The former possibility is more likely – that the chances of finding the 10,000th-most-used word would not be as low as 10,000-times less than the chances of encountering the most-used word.
As a language expands to include more words, it is likely that it issues a diktat to those words: “either be meaningful or go away”. And as the length of the language’s tail grows, as more exotic and infrequently used words accumulate, the need for those words drops off faster over time that are farther from Zipf’s domain.
Another way to quantify this phenomenon is through semantics (and this is a far shorter route of argument): As the underlying correlations between different words become more networked – for instance, attain greater betweenness – the need for new words is reduced.
Of course, the counterargument here is that there is no evidence to establish if people are likelier to use existing syntax to encapsulate new meaning than they are to use new syntax. This apparent barrier can be resolved by what is called the principle of least effort.
Proof and consequence
While all of this has been theoretically laid out, there had to have been many proofs over the years because the object under observation is a language – a veritable projection of the right to expression as well as a living, changing entity. And in the pursuit of some proof, on December 12, I spotted a paper on arXiv that claims to have used an “unprecedented” corpus (Nature scientific report here).
Titled “Languages cool as they expand: Allometric scaling and the decreasing need for new words”, it was hard to miss in the midst of papers, for example, being called “Trivial symmetries in a 3D topological torsion model of gravity”.
The abstract of the paper, by Alexander Petersen from the IMT Lucca Institute for Advanced Studies, et al, has this line: “Using corpora of unprecedented size, we test the allometric scaling of growing languages to demonstrate a decreasing marginal need for new words…” This is what caught my eye.
While it’s clear that Petersen’s results have been established only empirically, that their corpus includes all the words in books written with the English language between 1800 and 2008 indicates that the set of observables is almost as large as it can get.
Second: When speaking of corpuses, or corpora, the study has also factored in Heaps’ law (apart from Zipf’s law), and found that there are some words that obey neither Zipf nor Heaps but are distinct enough to constitute a class of their own. This is also why I underlined the word common earlier in this post. (How Petersen, et al, came to identify this is interesting: They observed deviations in the lexicon of individuals diagnosed with schizophrenia!)
The Heaps’ law, also called the Heaps-Herdan law, states that the chances of discovering a new word in one large instance-text, like one article or one book, become lesser as the size of the instance-text grows. It’s like a combination of the sunk-cost fallacy and Zipf’s law.
It’s a really simple law, too, and makes a lot of sense even intuitively, but the ease with which it's been captured statistically is what makes the Heaps-Herdan law so wondrous.
[caption id="attachment_24742" align="aligncenter" width="472"]
The sub-linear Heaps' law plot: Instance-text size on x-axis; Number of individual words on y-axis.[/caption]
Falling empires
And Petersen and his team establish in the paper that, extending the consequences of Zipf’s and Heaps’ laws to massive corpora, the larger a language is in terms of the number of individual words it contains, the slower it will grow, the lesser cultural evolution it will engender. In the words of the authors: “… We find a scaling relation that indicates a decreasing ‘marginal need’ for new words which are the manifestations of cultural evolution and the seeds for language growth.”
However, for the class of “distinguished” words, there seems to exist a power law – one that results in a non-linear graph unlike Zipf’s and Heaps’ laws. This means that as new exotic words are added to a language, the need for them, as such, is unpredictable and changes over time for as long as they are away from the Zipf’s law’s domain.
All in all, languages eventually seem an uncanny mirror of empires: The larger they get, the slower they grow, the more intricate the exchanges become within it, the fewer reasons there are to change, until some fluctuations are injected from the world outside (in the form of new words).
In fact, the mirroring is not so uncanny considering both empires and languages are strongly associated with cultural evolution. Ironically enough, it is the possibility of cultural evolution that very meaningfully justifies the creation and employment of languages, which means that at some point, languages only become bloated in some way to stop germination of new ideas and instead start to suffocate such initiatives.
Does this mean the extent to which a culture centered on a language has developed and will develop depends on how much the language itself has developed and will develop? Not conclusively – as there are a host of other factors left to be integrated – but it seems a strong correlation exists between the two.
So… how big is your language?
--
It all starts with Zipf’s law. Ever heard of it? It’s a devious little thing, especially when you apply it to languages.
Zipf’s law states that the chances of finding a word of a language in all the texts written in that language are inversely proportional to the word’s rank in the frequency table. In other words, this means that the chances of finding the most frequent word is twice as much as are chances of finding the second most frequent word, thrice as much as are chances of finding the third most frequent word, and so on.
Unfortunately (only because I like how “Zipf” sounds), the law holds only until about the 1,000th most common word; after this point, a logarithmic plot drawn between frequency and chance stops being linear and starts to curve.
The importance of this break is that if Zipf’s law fails to hold for a large corpus of words, then the language, at some point, must be making some sort of distinction between common and exotic words, and its need for new words must either be increasing or decreasing. This is because, if the need remained constant, then the distinction would be impossible to define except empirically and never conclusively - going against the behaviour of Zipf's law.
Consequently, the chances of finding the 10,000th word won’t be 10,000 times less than the chances of finding the most frequently used word but a value much lesser or much greater.
A language's diktat
Analysing each possibility, i.e., if the chances of finding the 10,000th-most-used word are NOT 10,000 times less than the chances of finding the most-used word but…
- Greater (i.e., The Asymptote): The language must have a long tail, also called an asymptote. Think about it. If the rarer words are all used almost as frequently as each other, then they can all be bunched up into one set, and when plotted, they’d form a straight line almost parallel to the x-axis (chance), a sort of tail attached to the rest of the plot.
- Lesser (i.e., The Cliff): After expanding to include a sufficiently large vocabulary, the language could be thought to “drop off” the edge of a statistical cliff. That is, at some point, there will be words that exist and mean something, but will almost never be used because syntactically simpler synonyms exist. In other words, in comparison to the usage of the first 1,000 words of the language, the (hypothetical) 10,000th word would be used negligibly.
The former possibility is more likely – that the chances of finding the 10,000th-most-used word would not be as low as 10,000-times less than the chances of encountering the most-used word.
As a language expands to include more words, it is likely that it issues a diktat to those words: “either be meaningful or go away”. And as the length of the language’s tail grows, as more exotic and infrequently used words accumulate, the need for those words drops off faster over time that are farther from Zipf’s domain.
Another way to quantify this phenomenon is through semantics (and this is a far shorter route of argument): As the underlying correlations between different words become more networked – for instance, attain greater betweenness – the need for new words is reduced.
Of course, the counterargument here is that there is no evidence to establish if people are likelier to use existing syntax to encapsulate new meaning than they are to use new syntax. This apparent barrier can be resolved by what is called the principle of least effort.
Proof and consequence
While all of this has been theoretically laid out, there had to have been many proofs over the years because the object under observation is a language – a veritable projection of the right to expression as well as a living, changing entity. And in the pursuit of some proof, on December 12, I spotted a paper on arXiv that claims to have used an “unprecedented” corpus (Nature scientific report here).
Titled “Languages cool as they expand: Allometric scaling and the decreasing need for new words”, it was hard to miss in the midst of papers, for example, being called “Trivial symmetries in a 3D topological torsion model of gravity”.
The abstract of the paper, by Alexander Petersen from the IMT Lucca Institute for Advanced Studies, et al, has this line: “Using corpora of unprecedented size, we test the allometric scaling of growing languages to demonstrate a decreasing marginal need for new words…” This is what caught my eye.
While it’s clear that Petersen’s results have been established only empirically, that their corpus includes all the words in books written with the English language between 1800 and 2008 indicates that the set of observables is almost as large as it can get.
Second: When speaking of corpuses, or corpora, the study has also factored in Heaps’ law (apart from Zipf’s law), and found that there are some words that obey neither Zipf nor Heaps but are distinct enough to constitute a class of their own. This is also why I underlined the word common earlier in this post. (How Petersen, et al, came to identify this is interesting: They observed deviations in the lexicon of individuals diagnosed with schizophrenia!)
The Heaps’ law, also called the Heaps-Herdan law, states that the chances of discovering a new word in one large instance-text, like one article or one book, become lesser as the size of the instance-text grows. It’s like a combination of the sunk-cost fallacy and Zipf’s law.
It’s a really simple law, too, and makes a lot of sense even intuitively, but the ease with which it's been captured statistically is what makes the Heaps-Herdan law so wondrous.
[caption id="attachment_24742" align="aligncenter" width="472"]
Falling empires
And Petersen and his team establish in the paper that, extending the consequences of Zipf’s and Heaps’ laws to massive corpora, the larger a language is in terms of the number of individual words it contains, the slower it will grow, the lesser cultural evolution it will engender. In the words of the authors: “… We find a scaling relation that indicates a decreasing ‘marginal need’ for new words which are the manifestations of cultural evolution and the seeds for language growth.”
However, for the class of “distinguished” words, there seems to exist a power law – one that results in a non-linear graph unlike Zipf’s and Heaps’ laws. This means that as new exotic words are added to a language, the need for them, as such, is unpredictable and changes over time for as long as they are away from the Zipf’s law’s domain.
All in all, languages eventually seem an uncanny mirror of empires: The larger they get, the slower they grow, the more intricate the exchanges become within it, the fewer reasons there are to change, until some fluctuations are injected from the world outside (in the form of new words).
In fact, the mirroring is not so uncanny considering both empires and languages are strongly associated with cultural evolution. Ironically enough, it is the possibility of cultural evolution that very meaningfully justifies the creation and employment of languages, which means that at some point, languages only become bloated in some way to stop germination of new ideas and instead start to suffocate such initiatives.
Does this mean the extent to which a culture centered on a language has developed and will develop depends on how much the language itself has developed and will develop? Not conclusively – as there are a host of other factors left to be integrated – but it seems a strong correlation exists between the two.
So… how big is your language?
Wednesday, 19 December 2012
NPPs in Japan
In the first general elections held since the phased shutdown of nuclear reactors across Japan, the Liberal Democratic Party (LDP) scored a landslide victory. Incidentally, the LDP was also the party most vehemently opposed to its predecessor, the Democratic Party of Japan (DJP), when it declared the shutdown of nuclear power plants (NPPs) across Japan, increasing the economic powerhouse's reliance on fossil fuels.
Now that Abe, who termed Noda's actions "irresponsible", is in power, the markets were also quick to respond. TEPCO's shares jumped 33 per cent, Kansai Electric Power's rose 18 per cent, the Nikkei index 1 per cent, and the shares of two Australian uranium mining companies rose at least 5 per cent each.
Now that Abe, who termed Noda's actions "irresponsible", is in power, the markets were also quick to respond. TEPCO's shares jumped 33 per cent, Kansai Electric Power's rose 18 per cent, the Nikkei index 1 per cent, and the shares of two Australian uranium mining companies rose at least 5 per cent each.
Wednesday, 5 December 2012
One of the hottest planets cold enough for ice
This article, as written by me, appeared in The Hindu on December 6, 2012.
--
Mercury, the innermost planet in the Solar System, is like a small rock orbiting the Sun, continuously assaulted by the star's heat and radiation. It would have to be the last place to look for water at.
However, observations of NASA's MESSENGER spacecraft indicate that Mercury seems to harbour enough water-ice to fill 20 billion Olympic skating rinks.
On November 29, during a televised press conference, NASA announced that data recorded since March 2011 by MESSENGER's on-board instruments hinted that large quantities of water ice were stowed in the shadows of craters around the planet's North Pole.
Unlike Earth, Mercury’s rotation is not tilted about an axis. This means areas around the planet's poles that are not sufficiently tilted toward the sun will remain cold for long periods of time.
This characteristic allows the insides of polar craters to maintain low temperatures for millions of years, and capable of storing water-ice. But then, where is the water coming from?
Bright spots were identified by MESSENGER's infrared laser fired from orbit into nine craters around the North Pole. The spots lined up perfectly with a thermal model of ultra-cold spots on the planet that would never be warmer than -170 degrees centigrade.
These icy spots are surrounded by darker terrain that receives a bit more sunlight and heat. Measurements by the neutron spectrometer aboard MESSENGER suggest that this darker area is a layer of material about 10 cm thick that lies on top of more ice, insulating it.
Dr. David Paige, a planetary scientist at the University of California, Los Angeles, and lead author of one of three papers that indicate the craters might contain ice, said, "The darker material around the bright spots may be made up of complex hydrocarbons expelled from comet or asteroid impacts." Such compounds must not be mistaken as signs of life since they can be produced by simple chemical reactions as well.
The water-ice could also have been derived from crashing comets, the study by Paige and his team concludes.
Finding water on the system’s hottest planet changes the way scientists perceive the Solar System’s formation.
Indeed, in the mid-1990s, strong radar signals were fired from the US Arecibo radar dish in Puerto Rico, aimed at Mercury’s poles. Bright radar reflections were seen from crater-like regions, which was indicative of water-ice.
“However, other substances might also reflect radar in a similar manner, like sulfur or cold silicate materials,” says David J. Lawrence, a physicist from the Johns Hopkins University Applied Physics Laboratory and lead author of the neutron spectrometer study.
Lawrence and his team observed particles called neutrons bouncing and ricocheting off the planet via a spectrometer aboard MESSENGER. As high-energy cosmic rays from outer space bombarded into atoms on the planet, debris of particles, including neutrons, was the result.
However, hydrogen atoms in the path of neutrons can halt the speeding particles almost completely as both weigh about the same. Since water molecules contain two hydrogen atoms each, areas that could contain water-ice will show a suppressed count of neutrons in the space above them.
Because scientists have been living with the idea of Mercury containing water for the last couple decades, the find by MESSENGER is not likely to be revolutionary. However, it bolsters an exciting idea.
As Lawrence says, “I think this discovery reinforces the reality that water is able to find its way to many places in the Solar System, and this fact should be kept in mind when studying the system and its history.”
--
Mercury, the innermost planet in the Solar System, is like a small rock orbiting the Sun, continuously assaulted by the star's heat and radiation. It would have to be the last place to look for water at.
However, observations of NASA's MESSENGER spacecraft indicate that Mercury seems to harbour enough water-ice to fill 20 billion Olympic skating rinks.
On November 29, during a televised press conference, NASA announced that data recorded since March 2011 by MESSENGER's on-board instruments hinted that large quantities of water ice were stowed in the shadows of craters around the planet's North Pole.
Unlike Earth, Mercury’s rotation is not tilted about an axis. This means areas around the planet's poles that are not sufficiently tilted toward the sun will remain cold for long periods of time.
This characteristic allows the insides of polar craters to maintain low temperatures for millions of years, and capable of storing water-ice. But then, where is the water coming from?
Bright spots were identified by MESSENGER's infrared laser fired from orbit into nine craters around the North Pole. The spots lined up perfectly with a thermal model of ultra-cold spots on the planet that would never be warmer than -170 degrees centigrade.
These icy spots are surrounded by darker terrain that receives a bit more sunlight and heat. Measurements by the neutron spectrometer aboard MESSENGER suggest that this darker area is a layer of material about 10 cm thick that lies on top of more ice, insulating it.
Dr. David Paige, a planetary scientist at the University of California, Los Angeles, and lead author of one of three papers that indicate the craters might contain ice, said, "The darker material around the bright spots may be made up of complex hydrocarbons expelled from comet or asteroid impacts." Such compounds must not be mistaken as signs of life since they can be produced by simple chemical reactions as well.
The water-ice could also have been derived from crashing comets, the study by Paige and his team concludes.
Finding water on the system’s hottest planet changes the way scientists perceive the Solar System’s formation.
Indeed, in the mid-1990s, strong radar signals were fired from the US Arecibo radar dish in Puerto Rico, aimed at Mercury’s poles. Bright radar reflections were seen from crater-like regions, which was indicative of water-ice.
“However, other substances might also reflect radar in a similar manner, like sulfur or cold silicate materials,” says David J. Lawrence, a physicist from the Johns Hopkins University Applied Physics Laboratory and lead author of the neutron spectrometer study.
Lawrence and his team observed particles called neutrons bouncing and ricocheting off the planet via a spectrometer aboard MESSENGER. As high-energy cosmic rays from outer space bombarded into atoms on the planet, debris of particles, including neutrons, was the result.
However, hydrogen atoms in the path of neutrons can halt the speeding particles almost completely as both weigh about the same. Since water molecules contain two hydrogen atoms each, areas that could contain water-ice will show a suppressed count of neutrons in the space above them.
Because scientists have been living with the idea of Mercury containing water for the last couple decades, the find by MESSENGER is not likely to be revolutionary. However, it bolsters an exciting idea.
As Lawrence says, “I think this discovery reinforces the reality that water is able to find its way to many places in the Solar System, and this fact should be kept in mind when studying the system and its history.”
Subscribe to:
Posts (Atom)