prasanna's posterous

qotd - each act, each occasion, is worse than the last, but only a little worse

From They Thought They Were Free: The Germans, 1933-45 by Milton Mayer. History repeats itself, be prepared.

"You see," my colleague went on, "one doesn't see exactly where or how to move. Believe me, this is true. Each act, each occasion, is worse than the last, but only a little worse. You wait for the next and the next. You wait for one great shocking occasion, thinking that others, when such a shock comes, will join with you in resisting somehow. You don't want to act, or even talk, alone; you don't want to 'go out of your way to make trouble.' … In the university community, in your own community, you speak privately to your colleagues, some of whom certainly feel as you do; but what do they say? They say, 'It's not so bad' or 'You're seeing things' or 'You're an alarmist.'

"These are the beginnings, yes; but how do you know for sure when you don't know the end, and how do you know, or even surmise, the end? On the one hand, your enemies, the law, the regime, the Party, intimidate you. On the other, your colleagues pooh-pooh you as pessimistic or even neurotic… the one great shocking occasion, when tens or hundreds or thousands will join with you, never comes. That's the difficulty. If the last and worst act of the whole regime had come immediately after the first and smallest, thousands, yes, millions would have been sufficiently shocked… But of course this isn't the way it happens. In between come all the hundreds of little steps, some of them imperceptible, each of them preparing you not to be shocked by the next. Step C is not so much worse than Step B, and, if you did not make a stand at Step B, why should you at Step C?"

qotd - ideology is a specious way of relating to the world

via finem respice farewell valci a quote from Vaclav Havel from his essay The Power of the Powerless. An essay that is well worth re-reading a few times. 

Ideology is a specious way of relating to the world. It offers human beings the illusion of an identity, of dignity, and of morality while making it easier for them to part with them. As the repository of something suprapersonal and objective, it enables people to deceive their conscience and conceal their true position and their inglorious modus vivendi, both from the world and from themselves. It is a very pragmatic but, at the same time, an apparently dignified way of legitimizing what is above, below, and on either side. It is directed toward people and toward God. It is a veil behind which human beings can hide their own fallen existence, their trivialization, and their adaptation to the status quo. It is an excuse that everyone can use, from the greengrocer, who conceals his fear of losing his job behind an alleged interest in the unification of the workers of the world, to the highest functionary, whose interest in staying in power can be cloaked in phrases about service to the working class. The primary excusatory function of ideology, therefore, is to provide people, both as victims and pillars of the post-totalitarian system, with the illusion that the system is in harmony with the human order and the order of the universe.

[...]

The post-totalitarian system touches people at every step, but it does so with its ideological gloves on. This is why life in the system is so thoroughly permeated with hypocrisy and lies: government by bureaucracy is called popular government; the working class is enslaved in the name of the working class; the complete degradation of the individual is presented as his ultimate liberation; depriving people of information is called making it available; the use of power to manipulate is called the public control of power, and the arbitrary abuse of power is called observing the legal code; the repression of culture is called its development; the expansion of imperial influence is presented as support for the oppressed; the lack of free expression becomes the highest form of freedom; farcical elections become the highest form of democracy; banning independent thought becomes the most scientific of world views; military occupation becomes fraternal assistance. Because the regime is captive to its own lies, it must falsify everything. It falsifies the past. It falsifies the present, and it falsifies the future. It falsifies statistics. It pretends not to possess an omnipotent and unprincipled police apparatus. It pretends to respect human rights. It pretends to persecute no one. It pretends to fear nothing. It pretends to pretend nothing.

Individuals need not believe all these mystifications, but they must behave as though they did, or they must at least tolerate them in silence, or get along well with those who work with them. For this reason, however, they must live within a lie. They need not accept the lie. It is enough for them to have accepted their life with it and in it. For by this very fact, individuals confirm the system, fulfill the system, make the system, are the system.

qotd - we are always living in expectation of better things

From that curmudgeon of a philosopher Arthur Schopenhauer in his essay the emptiness of existence. While Schopenhauer mostly comes off as pessimistic there is a certain truth and beauty in his words that shines through.

The scenes of our life are like pictures in rough mosaic, which have no effect at close quarters, but must be looked at from a distance in order to discern their beauty. So that to obtain something we have desired is to find out that it is worthless; we are always living in expectation of better things, while, at the same time, we often repent and long for things that belong to the past. We accept the present as something that is only temporary, and regard it only as a means to accomplish our aim. So that most people will find if they look back when their life is at an end, that they have lived their lifelong ad interim , and they will be surprised to find that something they allowed to pass by unnoticed and unenjoyed was just their life — that is to say, it was the very thing in the expectation of which they lived. And so it may be said of man in general that, befooled by hope, he dances into the arms of death.

Then again, there is the insatiability of each individual will; every time it is satisfied a new wish is engendered, and there is no end to its eternally insatiable desires.

This is because the Will, taken in itself, is the lord of worlds; since everything belongs to it, it is not satisfied with a portion of anything, but only with the whole, which, however, is endless. Meanwhile it must excite our pity when we consider how extremely little this lord of the world receives, when it makes its appearance as an individual; for the most part only just enough to maintain the body. This is why man is so very unhappy.

qotd - diamonds are forever

From The Rise and Fall of Diamonds by Edward Jay Epstein. The success of diamonds in modern society is really the triumph of marketing. I've never really understood why people follow certain rituals like diamond rings for engagement but then again there are many norms that make no sense to me. What fascinates me is taking something that is unnecessary to a given by manipulating how people feel and give them an illusion to aspire to. I remember being at a dinner a few years ago when one guy told another guy he hadn't spent enough money on the ring he gave his fiancee. I just sat there laughing in my head but then it dawned on me this social proof aspect to diamond engagement rings creates a great deal of pressure on people. The same amount of money could be allocated to something else. Most of this conspicuous consumption is geared towards a signaling mechanism to other people to say we are superior simians. 

Then, in 1967, halfway around the world, a South African diamond company decided to change the Japanese courtship ritual. It retained J. Walter Thompson, the largest advertising agency in the world, to embark on a campaign to popularize diamond engagement rings in Japan. It was not an easy task. Even the quartering of millions of American soldiers in Japan for a decade had not resulted in any substantial Japanese interest in giving diamonds as a token of love.

The advertising agency began its campaign by subtly suggesting that diamonds were a visible sign of modern Western values. It created a series of color advertisements in Japanese magazines showing very beautiful women displaying their diamond rings. The women all had Western facial features and wore European clothes. Moreover, in most of the advertisements, the women were involved in some activity that defied Japanese traditions, such as bicycling, camping, yachting, ocean-swimming and mountain-climbing. In the background, there usually stood a Japanese man, also attired in fashionable European clothes. In addition, almost all of the automobiles, sporting equipment and other artifacts in the picture, were conspicuous foreign imports. The message in these ads was clear: diamonds represent a sharp break with the Oriental past and an entry point into modern life.

The campaign was remarkably successful. Until 1959 the importation of diamonds had not even been permitted by the postwar Japanese government. When the campaign began in 1968, less than 5 percent of Japanese women getting married received a diamond engagement ring. By 1972 the proportion had risen to 27 percent. By 1978, half of all Japanese women who were married wore a diamond on their ring finger. And, by 1981, some 6o percent of Japanese brides wore diamonds. In a mere thirteen years, the fifteen-hundred-year Japanese tradition was radically revised. Diamonds became a staple of the Japanese marriage. And Japan became, after the United States, the second largest market for the sale of diamond engagement rings. It was all part of the diamond invention.

The diamond invention was an ingenious scheme for sustaining the value of diamonds in an uncertain world. To begin with, it involved gaining control over the production of all the important diamond mines in the world. Next, a system was devised for allocating this controlled supply of gems to a select number of diamond cutters who all agreed to abide by certain rules intended to assure that the quantity of finished diamonds available at any given time never exceeded the public's demand for them. Finally, a set of subtle, but effective, incentives were devised for regulating the behavior of all the people who served and ultimately profited from the system.

The invention had a wide array of diverse parts: these included a huge stockpile of uncut diamonds in a vault in London; a billion-dollar cash hoard deposited in banks in Europe; and private intelligence network operating out of Antwerp, Tel Aviv, Johannesburg and London; a global network of advertising agencies, brokers and distributors; corporate fronts in Africa for concealing massive diamond purchases; and private treaties with nations establishing quotas for annual production.

The invention is far more than merely a monopoly for fixing diamond prices; it is a mechanism for converting tiny crystals of carbon into universally recognized tokens of power and romance. For it to ultimately succeed, it must endow these stones with the sort of sentiment that would inhibit the public from ever reselling them onto the market. The illusion thus had to be inculcated into the mass mind that diamonds were forever-- "forever" in the sense that they could never be resold.

qotd - financial hackery (or how greece and italy performed tobashi)

From Extreme Money about how Greece and Italy both hid debt using some creative financial hackery. If there is a need the market will find a way to cater for it. The only problem is sometimes that pesky thing called reality sets in. Reality needs to go away ;)

In the 1990s, Japanese companies and investors pioneered the use of derivatives to hide losses, a practice called tobashi (from the Japanese verb tobasu meaning “to make fly away”). Subsequently, a number of European countries used derivatives to disguise their borrowings. 

Derivatives, such as interest rate and currency swaps, are normally used to alter the nature and currency of cash flows on assets or borrowings. Transactions involve exchanging one stream of payments for another. At commencement, if the contract is priced at current market rates, then the value of the two sets of cash flows should be roughly equal. 

Using artificial or “off-market” interest or currency rates, it is possible to create differences in value between payments and receipts. If the value of future payments is higher than future receipts, then the party making the future payments receives an up-front amount reflecting the positive value of the contract. In effect, the participant receives a payment today that is repaid the higher than market payments in the future—identical to a loan. Figure 13.3 sets out the strategy using a simple currency swap. 

Screen_shot_2011-12-01_at_12

In 2008, Greece was found to have entered into a series of transactions with Goldman Sachs to disguise its debt. Earlier, academic Gustavo Piga identified an unnamed European country, generally assumed to be Italy, using derivatives to provide similar window dressing. 

In December 1996, Italy allegedly used a currency swap against an existing ¥200 billion bond ($1.6 billion) to lock in profits from the depreciation of the yen. Done at nonmarket rates, the swap was really a loan where Italy accepted an unfavorable exchange rate and received cash in return. The payments were used to reduce Italy’s deficit, helping meet the European Union target of less than 3 percent of GDP under the EU Maastricht Treaty. 

The Greek transactions were similar off-market cross-currency swaps linked to the country’s foreign currency debt. The swaps were for a notional principal of approximately $10 billion, with maturities between 15 and 20 years. The cash received may have reduced the country’s debt/GDP ratio from 107 percent in 2001 to 104.9 percent in 2002 and lowered interest payments from 7.4 percent in 2001 to 6.4 percent in 2002. The future payment obligations under the swaps were not reported as a future liability for Greece. 

Dealers made money by helping clients hedge or take on more risk using derivatives. Clients also paid dealers handsomely for structures that took advantage of regulatory loopholes. Goldman Sachs allegedly made around $300 million from the Greek job.

qotd - distilling modern consumerism

From Extreme Money: Masters of the Universe and the Cult of Risk. A little quote from Vance Packard on modern consumerism. It's quite sobering to sit back and think about how much we're duped by an advertising machine that seeks to exploit our insecurities in exchange for nothing more than a vague promise that the items we buy will validate our existence. Of course that validation is immediately negated once the next best thing comes around. Then it's time to level up with all the cool kids.

In his three best-selling books The Hidden Persuaders (1957), The Status Seekers (1959), and The Waste Makers (1960), Vance Packard, an American journalist, highlighted the use of psychological techniques to manipulate consumers. Status and fear of loss of status was used to sell goods. Planned obsolescence increased demand for products, long before they required replacement.

A toothbrush does little but clean teeth. Alcohol is important mostly for making people more or less drunk. An automobile can take one reliably to a destination and back.... There being so little to be said, much must be invented. Social distinction must be associated with a house...sexual fulfillment with a particular...automobile, social acceptance with...a mouthwash [and so on]. We live surrounded by a systematic appeal to a dream world which all mature, scientific reality would reject. We, quite literally, advertise our commitment to immaturity, mendacity and profound gullibility. It is the hallmark of our culture. 

unshred - my solution to instagram's engineering challenge

I came across the Instagram Engineering Challenge: The Unshredder on hacker news the other day. So I thought I'd sit down try and come up with a quick and dirty way to solve it. In this case a naive solution seemed to be the way to go and it's not horribly slow (I'm sure most engineers would laugh at the way I solved it and there's probably a much cleaner way to do ;).  My solution works for the use case they gave but breaks down when the width of a shred is less than 8. I also implemented the bonus problem which was to automatically detect the width of each shred (the default image is 32). Overall I enjoy programming challenges like this since it only takes a few hours and gives you a chance to try something different. For employers this is a good measure if someone codes for fun but given I make a terrible employee I'm probably not the sort of the person they're aiming for. I'm also a big fan of employers giving candidates homework projects to work on at home that take at most a weekend a complete. You get a decent sense of how people solve problems plus if they have some kind of work ethic. It adds an extra dimension beyond regular interviews.

1. Understanding the problem

The idea is to go from a shredded image to the original given a particular width of n to it's original form

Untitleddrawing

My first intuition was comparing the edges of shreds. Shreds that are next to each other should have very similar pixel colors for the most part. That is for two given shreds S1 and S2 that are next to each other the left edge of S1 should be similar to the right most of S2. We can use the euclidean distance function to calculate the distance between two colors. It's probably better to average a window of colors than compare every pixel to every other. After trying a few different window sizes I settled on 8 which produced good results (except for shred widths less than 8).

Actual adjacent blocks exhibit much more similar color patterns versus a randomly place shred.

Untitleddrawing_1

They were kind enough to provide some starter code for manipulating PIL and generating shredded images. I went and built a set of images based on different shred sizes so I could test across different shreds. I also tried it with a few other pictures of the same size as the source image with different content. 

2. Unshredding

You can see the attached source code for the implementation details but the basic algorithm roughly works as follows

1. for each shred S find the adjacent shred S' using 2.

2. for a given shred S compare it every other shred S' to find the one that is most similar to it
  • to compare shreds for similarity average 8 pixels at a time S(LEFT EDGE) and S'(RIGHT EDGE)
  • for each window of averaged pixels calculate the euclidean distance between them
  • for each distance which is closer than some threshold (default is 32) increment the score
  • if the score is higher for this S' than the last high score for this S save the pair of S' and the score
3. store in map SM[S'] = S 

4. keep track of the lowest score for each S and store the S as the end_shred. 

5. starting at the make current_shred = end_shred
  • copy the shred from the orginal image to next right shred position
  • the current_shred will be SM[current_shred]
  • stop when you've done all shreds
3. Shred Width

To calculate the shred width we want to see what which shred widths that the image supports have the most dissimilarity

1. from min shred width to the max shred width
  • check each shred and calculate the dissimilarity (height/window size - similarity score ) between the shred and the shred next to it (use the compare function from first algorithm)
  • average the dissimilarity scores from above if it's great than the last stored dissimilarity score store the width
  • you can optimize and skip higher widths which are multiples  width with highest dissimilarity if they have the same dissimilarity score

 

 

Click here to download:
instagram_unshred.py (6 KB)