With the Internet, We're All Street Performers

There are many articles online that echo the MPAA talking points about PIPA and SOPA, the Senate and House versions of a law that gives media companies the right to command law enforcement to go after those they target with little or no judicial review, no requirement of evidence, no right of the victims to challenge their accusers.

For me, these articles all do away with any authority they could have brought to the issue by attempting to associate those who download MP3s and MP4s as "thieves." The thieves they try to remind you of are those breaking into someone's home, or taking something off your person. Some articles go even farther and try to compare downloading to drug trade, or even terrorism.

Let's be clear in our terminology. Downloading an MP3 is a form of "theft," but not that kind. In order to take something from you or your house, I need to deny you access to it. In order to download an MP3, I take nothing - I enjoy your efforts without paying you, like someone passing a street performer without putting anything in the hat.

There are no laws stating you owe a street performer, even if you stick around while they play. But media companies don't see themselves this way either because they're full of themselves, or they see the end of that legal road, and don't want to go there.

To be concrete, Louis CK, Trent Reznor, Radiohead, and several more artists have posted their work for free or variations (sometimes a small flat price, sometimes you pay what you want etc), and made plenty of money anyway. I haven't heard them outwardly acknowledge they're doing a global street performance, but the business model is identical: Some paid. Some didn't. The artist did well. At its core this is how media works - you put it out there and you hope some people like it enough to pay for it. Some like it and don't pay. If you're unlucky, few like it in the first place.

Implicit in this exchange are the ideas of availability and value. On value, the consumers who pay do so in part because they value the work and want to do their part to see more like it made in the future. The consumers who don't may have many reasons, but at least part of that reasoning is that they view the cost as too high for the work.

On availability, media companies are stuck in a circular argument in which they've managed to jail themselves. Media companies apply region protection to their distributions. They'll release a movie first in the US, then take it to DVD and on those DVDs they'll apply a Region Code that locks those DVDs to the US. Then they'll license that movie to Netflix and in that licensing contract, lock distribution to only US. Later, if there was enough revenue in the US, they might target global markets.

During the time between initial release and global release there are international customers, many who would likely pay for it if they could. The internet has no region codes, no licensing restrictions - and so many download it for free, in part because it's their only option. In return, media companies increase their accusations and fear of international customers and increase their attempts at restricting regional distribution in a confused desire to keep their content from reaching international shores.

I'll pause for those skeptical - you might point out that media companies must do things this way for a reason, and there's no proof it would get better if they got rid of these cave-tech region restrictions. But remember that the artists doing their own releases above did not region-restrict their media, and had numerous international customers. For a larger example, I point to a company that has made a business model out of global release, a video game distributor: Valve. Valve created a platform called Steam they use as a global marketplace to sell games. While the Steam platform warrants a lengthy analysis of its own, let's stick to the one point for now: Global availability. Valve has seen piracy drop dramatically in migrating to a global release model. Don't take my word for it - here's founder Gabe Newell on how this change has made some unexpected markets like Russia some of their best places for sales.

Inside the US, international availability is a minor issue (some travel enough to not want the irritation of a region-restricted DVD), but in many cases convenience is the issue instead - in this case the availability issue is their living room, not their country. I can go on a torrent site right now and download every movie and TV show I've ever heard of. I can go on Netflix, Hulu, Amazon, etc and get a randomly changing slim slice of that same selection. Shouldn't I start where the selection is greatest?

And frankly in many cases the value just isn't there. Big movie companies think they should be getting $10 at the box office - now that they're adding the 3D gimmick, they think they should be getting $15-20. Millions of downloaders do so in part because they disagree.

We don't need new laws that force those consumers into compliance with what these content distributors feel they're owed, and force consumers to wait until they have the privilege of availability. Both of these are attempts at perverting the market. If Congress really wants guidance on how to write a new law, begin by doing away with the worst parts of the DMCA, where for example free speech can be squashed by making a false copyright claim. Then, if more protections are needed - and perhaps they are - look at Valve and Louis CK's business model, and ask them what would help them, not Viacom and Rupert Murdoch.

We all hope to create content at some point in our lives that others enjoy, and hopefully even pay for. But the big businesses are trying to force us back into an old model and aren't ready to accept that in this new global market place where media moves so readily, we're all street performers. Even the big guys.

How To Convert All Text To Lowercase In Google Spreadsheets

We consume a lot of table-like data from clients that has a lot of human error in it. Often this is big enough that manual fix-up isn't fast enough (more than 100) but not so big that some more elaborate solution is appropriate (millions of entries needing formulas and machine learning). It's just a static table of data that needs to get loaded into a Dictionary<string, string>.

When doing so we dump the data into Google Spreadsheets, perform some fixups, and then share it with the client to verify it's accurate. One annoyance is there's no way to just switch a column to lowercase directly, so here's one way to pull it off:

1. Select the column, right-click, and click Insert Column Right.
2. In the first cell of the new column enter formula =LOWER(A1) (or whatever the first cell is of the original column).
3. Copy the cell with the formula in it.
4. Select the column and paste - Google Spreadsheets is smart enough to adjust the row number for each pasted entry. You now have a column with lowercase in it - but it requires that column to the left to still be there.
5. Select the new column again and copy.
6. Right-click and click Paste Special > Values only.

This wipes out the formulas. You can now delete the original column. Obviously this applies to anything you could do with a formula in spreadsheets.

NoSQL - Where's It Going? Where Should It Go?

The NoSQL movement is either saving web platforms or a major nuisance, depending on what kind of developer you happen to be. Either way, the way we store data is shifting. There are the old stand-by Relational DBs like MySQL, Oracle and SQL Server, and then there's all the crazy new wave stores - BigTable, HBase, Cassandra, SimpleDB, etc, all falling under the general category of "NoSQL."

Each of these has their design features and focus, and have less in common than maybe an umbrella like NoSQL should allow. Then again, by defining them by what they are not, I suppose a ham sandwich could also be eligible for the NoSQL category.

In the simplest cases, all of these solutions allow you to get a basic job done: store rows of similar-ish data in a list. After that, things get crazy.

The first general category they differ by is performance. Some favor write performance. Some favor low-latency consistency. Some favor read performance. Some favor availability. It's unfortunate my choice of data store for my entire app impacts these factors. Really these are all strategies I'll need in varying amounts for different tasks my app performs.

The second general category they differ by is how you read and write data. Each generally has its own new-fangled API for accessing it. Some have libraries that let you pretend you're still using SQL, and tend to throw a lot of errors when you do anything interesting, let alone fancy. For example, Amazon's SimpleDB, while indeed simple, cannot handle relating data between 2 tables (which it calls domains). While it has a SQL-like interface that sits on top of its API, most of the SQL you're used to using will throw an error (like JOIN). What a nice cage to build an app inside of.

This failing of SimpleDB and some of the other NoSQL options seems to be from a bit of confusion about what NoSQL means. Although it generally means it's not a Relational Database, that doesn't mean data never has relationships. It simply means that it has no explicitly defined relationships in the database itself - really what people want is to never think about a Foreign Key again in their lives. Put another way, relationships are business logic that belong in code, not the database. Fetching data and how that's performed is the responsibility of the data provider, not the app.

It seems there needs to be a general standard for NoSQL databases that's defined by what it is rather than what it is not, and here's what I see developers really looking for:

• Named tables
• Allows you to submit arbitrary data into tables ("schemaless")
• Does not enforce data relationships (not a relational DB), but...
• Allows you to join tables in queries
• Allows sorting and filtering
• Scalable

That's what people are really looking for in a NoSQL database. It eliminates the upfront cost of schemas, and eliminates a lot of the performance cost of storing all those rows in a scalable way. The one big burden hanging out there is handling joins - but it's still something that can be accomplished with scalability in mind, and it can be done at the data layer so a master in the data provider can service requests for any data of any shape.

I put scalable last because the truth is that a lot of apps being built with NoSQL solutions are just hopeful. They have no need for something more scalable than a typical MySQL or even SQL Server Express instance can provide. But they do want to be done with schema management, and they want to design their app so it can handle the big time if it gets there.

There are some further features I'd like to see ideally, but don't have to be there to fulfill the basics of what these modern DBs ought to be:

• Ideally: Allows querying with SQL
• Ideally: Handles indexing of multiple columns, preferably in response to queries
• Ideally: Lets you specify the strategy for a specific table's storage ("the engine" by today's terms)
• Ideally: Handles sharding etc strategies for you so you can store all data in a single named table, even if it's broken down into many smaller tablets under the hood

I don't have a great name for where these solutions all seem to be headed. Schemaless Joinable Tables?

Google's Dart

Google's Dart looks pretty cool.
http://www.dartlang.org/docs/getting-started/

It borrows (and improves) the only thing I like from PHP - shorthand for variables and expressions in strings:
'Hello $name' replaces $name with the value of the name variable.
'Answer: ${a + b}' performs the expression a + b and swaps that result in.

It also allows both var and typed variables in the same app like Javascript and C#, and borrows one of C#'s best features, Lambda Functions:

num circumference(num r) => r * 3.14;

Finally, it doesn't have a heavy focus on making things private - nothing is private by default (you instead make something private by prefixing its name with an underscore). This is a somewhat odd syntax, but I think one of Javascript's enduring and under-recognized strengths is that everything is by default public. This makes Monkey-Patching broken-but-useful libraries possible, something that's impossible in Java and has caused enormous amounts of pain in numerous past Java projects I've worked on.

Looking forward to seeing where Dart goes next. My suggestion: Port the Closure library to Dart.
http://code.google.com/closure/library/

Surviving Google's Blogpocalypse

I attempted to login to my Google Apps version of Gmail one day and was instead presented with a page I couldn't circumvent. Over 20 checkboxes, several tabs that didn't look like tabs, and a lot of confusing options. Eventually I was able to access Gmail again.

The next time I attempted to login to Blogger however, things went poorly. As it turns out this transition does not have a migration path for Blogger/Blogspot, so you have to migrate manually.

Migrating manually is not obvious or easy. Here's the steps I had to take - maybe they'll help others:

1. Choose the option to create a new personal Gmail account for your blog. You'll need to go through the usual Gmail signup process where you create another username, another password to remember, and have to enter another arbitrary security question.
2. Login with the new Gmail account. You can signout of your existing account, or do this in an Incognito window or another browser to skip the logout step.
3. Now you have access to your blog again! ...but you don't want to have to use this other random Gmail account to edit it every time.
4. Get to Settings > Permissions
1. In the old look: Under the blog name, click Settings, then the Permissions tab.
2. In the new look: Click the blog name, click Settings, and look under Permissions.
5. Click Add Authors, and add your old Apps account - except - you can't just add it normally - the operation just quietly fails with no open invite, no indication the invite went out, no email, and no indication of an error. Instead, you need to use an alias for your account. If you have multiple domains associated with your account this is simple - use one of the alias domains. If you don't, you'll need to create an Alias for your user in the Domain Admin, then invite that Alias.
6. Check your email for the invite, and accept.
7. Come back to the incognito window (or, sadly, logout and login as the ephemeral Gmail account you created). Get to Settings > Permissions again, and now change your invited self to Admin instead of Author.
8. You can finally return to using Blogger the way you always did before Google ruined your day.

I'd like to point out that this transition really should have been entirely under the covers - as a user, I login with my Google Apps email address to edit my blog. I want to keep doing so. That Google is going through a major systems changeover shouldn't require me going through all of this trouble. If the transition is going to be mandatory, it should have waited until all products could be migrated automatically. Instead a lot of non-technical users had to deal with this insane process that offers no support.

Google is notoriously poor at customer support. I actually recall standing at TGIF listening to a Googler ask the founders why customers are pushed to post their problems to Support forums no one reads, getting no response unless an employee happens to take it upon themselves to look into it, or it makes it onto the front page of Slashdot. Sergey Brin's actual response was, "Well we shouldn't resolve these issues by having a big customer service department. We should resolve them by writing better code." I really should've grabbed a mic and described a metaphorical situation in which a farmer starts closing the barn door after his cow wanders off, but alas.

This is the support thread for this problem: It's safe to assume no Google employee will ever respond to it, let alone read it. http://www.google.com/support/forum/p/blogger/thread?tid=239869e385664e6b&hl=en&fid=239869e385664e6b0004acf9193ad5a4

Kintera.org/Blackbaud.com infecting its users - on its donation page

I recently tried to donate money to a friend's charity. The page is hosted on Kintera.org, which includes a form to collect credit card info, and a Java applet that shows who else has donated recently. It uses a scrolling library they probably pulled off some untrustworthy website (I doubt it's the worse possibility - Kintera willfully infecting those making donations).

Unfortunately that scrolling library has 3 viruses, all of which act as Trojans to infect the user's machine and place them at the whim of a command and control bot network:

Java CVE-2008-5353.KM
Java CVE-2009-3867.GC
Java CVE-2008-3869.M

That's pretty embarrassing. The scroll page actually shows one page before you fill out your credit card info, so in the absolute worst case scenario, you view the page, click Continue while the infection is occurring, a keylogger downloads and runs, you enter your credit card info, and off it goes to as many as 3 bot network owners/users. Not cool.

Confidence indeed.

How to Root the HTC Evo Shift 4G

Sprint blocks their forums from viewing by non-logged-in users; this same information is posted at:
http://community.sprint.com/baw/message/329584

But you probably can't view it. Here it is reposted: How to root the HTC Evo Shift 4G.

You need the JDK installed:
http://www.oracle.com/technetwork/java/javase/downloads/java-se-jdk-7-download-432154.html

The Android SDK installed:
http://developer.android.com/sdk/index.html

And the HTC Sync software installed:
http://www.htc.com/www/help/ (scroll down to HTC Sync for all HTC Android phones and click Download)

Now follow these instructions:
http://forum.xda-developers.com/showthread.php?t=1185243

You'll need to cd into the directory where the Android SDK was installed, and then into the platform-tools directory inside that, in order to run adb and perform the other commands they ask you to run. You also need to move the 3 files they tell you to download into platform-tools (or, reference the path you downloaded them to in the commands you run - adb push).

This works on the current version as of this posting Aug 27, 2011: Android 2.3.3, but is unlikely to work in a future OTA update if there is one. Note that this only gives you temporary root but that's all you need to wipe out built-in apps you don't want. Note also that other temp root solutions like Visionary and permanent root solutions like ShiftRR will not work. Only the method linked to above will work on this latest OTA.

You can easily delete built-in apps while rooted by installing ES File Explorer from the Market (it's free), then go into Menu>Settings and check Root Explorer, then check Mount File System. Then browse to /system/app (you may need to change Home Directory to / instead of /sdcard to get to it). Press and hold on built-in apps you don't want, then tap Delete.

I deleted Amazon MP3, Nascar, NFL ("sfl-prod-release.apk"), Sprint Navigator, Sprint TV, and Swype (so I could install the latest). I doubt it's smart to get rid of the annoying Sprint Zone app because it appears to be how PRL updates etc get onto the phone.

You can prevent future OTA updates from putting all these apps back on by tapping Menu>Settings>Software Updates>HTC software update and uncheck Scheduled check. You can always explicitly ask for an OTA update if you want by coming back to this screen and tapping Check now.