I hate to tell you this, but it has NOTHING to do with technology or even individual agents. The closed MLS has everything to do with politics. Period.
The biggest brokers in an area will monopolize 60%+ of all the data in a single MLS. When they have control over the listings, they have control over the MLS. If they threaten to leave an MLS, the MLS is worthless, and one company wins.
So, the MLS compromises. It does the bidding of the biggest listing offices and everyone else suffers.
I speak from first hand experience as a vendor in multiple markets. An open mls would be mind blowing and wonderful, bit it's not a matter of setting up a nice website, it's a matter of beating through the bureaucracy.
I've spent a lot of time thinking about this problem, and I independently arrived at the same conclusion. I know several agents and have spent a lot of time discussing the politics around the system.
This is absolutely NOT a technical problem. The real problem has several facets:
1. Politics. You covered this.
2. Agent resistance to change. This is a huge issue. Roughly 80% of agents I've encountered are tech neophytes and can hardly use their own existing MLS system, which hasn't changed in a decade. Technologists look at Web 2.0 as being an ease-of-use panacea, but this is a group of users that literally run away from anything new or different, especially technology. Not every agent falls in this category, but most do.
3. Most users see the existing MLS as good enough. Web nerds cringe when they see circa-1996 web UI and fully closed data silos. Agents see a system that technically gets the job done, even if it's a bit cumbersome. If you ask an average agent what his or her work-related biggest pain points are, the MLS UI will not be in the top 5.
You make an interesting comment about the larger brokers. In my area there is one that is, relatively speaking, bigger than the other 'big fish', but there are probably three or four "significant" brokerages, and a lot of smaller ones too.
What is it about the market that causes nominally entrepreneurial people (they're commission-driven after all) to tie themselves to a large brokerage that will eat a lot of their earnings?
Seems like the same political problems that are in place with Ticketmaster. Perhaps if someone figure out how to beat through this bureaucracy they can figure that one out, too.
There's actually a startup that's local to me that's fighting that fight.
What's interesting is that they haven't tried to go after big names at first... it's surely more of a long tail kind of thing. They started off only doing small, local venues, but they're doing so well that they're naturally starting to pick up bigger and bigger ones.
I'll be excited to see Ticketmaster go. I don't know of anynone who enjoys dealing with them.
I've seen the effects of this monopoly even before the Internet age, and this is a pretty accurate article. The data on the MLS is not rocket science, it could be trivially reimplemented in a lot better ways than it has been.
In defense of real estate agents, however, they aren't necessarily willing participants in this fiasco. For instance, a city's real estate board can charge agents fees (and really, really, really ridiculous fees) for access to the service. And there's no choice: it's the only way to be a realtor. Agents suck it up like just another cost of doing business, but the reality is that the real estate boards are acting extremely unethically.
"And there's no choice: it's the only way to be a realtor."
Important nitpick: Realtor is a registered trademark of the National Association of Realtors. Not all real estate agents are Realtors. It's yet another racket in the industry.
As a real estate agent, I agree with the author's point, especially about the technology. I not only pay hundreds of dollars per year, but technologically they still can't even keep up with basic advancements such as making the mls work in firefox or safari. The problem, as far as I can tell, is not from individual agents trying to protect their racket, but from our regional and national associations and their somewhat opaque partnerships with private businesses.
For example, our national association doesn't even own Realtor.com; they made an apparently permanent agreement with a private company which charges us additional fees for "enhanced" listings.
The other problem is that the MLS's are all controlled regionally, and it's not so much about the technology as it is about the bureaucracy. While the author makes an excellent point, many are currently attempting it but it's not easy. Companies such as Redfin seem to be slowly getting there. As an agent, I'd be the first to support a free, nationwide MLS, but I don't know if it's something that can be solved by technology alone.
As someone who worked for an agency supplying a few hundred realtors across 6 different MLSes with web development, I can also testify to the utter technical incompetence of at least one of the major suppliers (who shall remain unnamed) of back-ends for hundreds of MLSes.
I noticed a bug in our CSV import one day and I discovered that they were quoting, but not escaping quotes in any way. Thereby making it possible for any realtor to inject whatever the data they wanted for anyone else's listings either before or after theirs (depending on how the data importers resolved conflicts).
This was a huge security hole affecting probably hundreds of thousands of listings nationwide, and you know what their developer support said to me? They actually sent me a suggested workaround tailored to the particular instance of corrupted data that I had encountered. When I explained to them the ramifications of this to their business they simply ignored me.
I handle listing data ingestion and distribution at HotPads.com, so I deal with some of these problems all the time. There hasn't been a single complete game changer in this space yet, and with the way things are currently progessing I think the solution to the MLS problem will come as a combination of MLSes learning to play nice (we see quite a few distributing their data across multiple sites) and agents and brokers signing up with distribution services, listing their properties on more open sites, or handling their own distribution directly. We have listing feeds with examples of all of these cases.
We are in the real estate and have been pondering this problem for years.
It is not a technical problem, and there is no technical solution.
It's easy to make a better database, but it isn't useful if it isn't full of data. And you won't get lots of data because the network effects are too strong. It has been tried many times.
Besides that, you must realize that MLS's make most of their revenue from MLS fees; if they didn't run the MLS then they wouldn't have a very big budget, and many people would lose their jobs. MLS's have strong relationships with their members (agents and brokers). There is no incentive for anyone close to the data sources to support such a thing.
The MLS pains are a symptom of the broader question, "do we still need human realtors?"
Sadly, Foxtons US is dead (http://www.reuters.com/article/bondsNews/idUSN27422652200709...) and ZipRealty has been around for 7+ years without revolutionizing the industry. There must be a deeper explanation than "Realtors make 6% a sale for accessing a database."
My hypothesis is "rational, well-informed market decisions" would ensure few purchases and sales ever close. With so much on the line (life savings, huge debt, negotiation anxieties, moving uncertainties), normal people simply can't handle the stress without some hand-holding.
As a licensed agent, I've personally had plenty of cases where I knew I found the clients a steal, but I had to spend enormous amounts of energy and time convincing them take action (and before someone else beats them to it).
The National Association of Realtors will and has done everything to protect the MLS, which they manage and make millions of dollars from. The NAR is the largest American trade union and 3rd largest lobby
"Make this extra-easy via functionality that auto-populates MLS when an agent lists a property on OSLS."
This violates my local MLS' TOS. Pretty much everything violates the MLS TOS. Did you know that agents can't put their contact info on their MLS listings?
What about the idea of making it easier to sell a house using a "For Sale By Owner" then let more ppl list their own homes on craigslist or wherever they want. If I was trying to sell my home I would want it on every single site out I could get it on. It seems like what would aid this is if there were about 10 standard contracts that everyone agreed to use. You could research it and know all about the type of contract you were going to use - about like learning a software license. Then any more give and take is just worked out in the price.
There are many companies that offer such a service today, and it is starting to help. Syndication is what will eventually allow the MLS monopoly to be broken I think...
Five years ago probably 95% of homes found by buyers online were found via MLS search sites. But syndication has started to shift this number lower and distribute it among many other sites (trulia, zillow, craigslist, etc etc).
As the % of online shoppers viewing true MLS data drops, brokers will continue to syndicate their listings to other sites, which will accelerate the shift away from MLS reliance even further.
After a while, the monopoly will be broken and then you will start to see radical shifts like you describe where real estate agents are used for their professional services -- advice, negotiation, paperwork, and expertise, and NOT as an overpriced access fee to the MLS data.
I think this idea is interesting. I think there's a real trend of the internet systematically destroying information gatekeepers; it makes sense, as the spread of information harms those who rely on the scarcity of information to make a profit.
It's sort of like applying an open expert system to everything.
I think this value proposition, "We make it easy to navigate x beaurocracy" is pretty interesting.
I recently heard about the business of teaching people how to form syndicates to buy "short sale" homes and then flip those homes for a profit. (Short sale == sell house for less than the loan value.) If you pay $$ to a guru you can get standardized contracts, hand holding, etc. Example blog post from someone in this area:
http://cawholesaledeals.com/blog/tag/short-sale-syndication/
Leaving aside the morality of these gurus...
Wonder if this would be a thin edge of the wedge area to start with aggregating data on house sales? People are already meeting in person and swapping information about where are the sales, which ones do they want to syndicate, so you have less of a bootstrapping problem. You could also provide ways for people to signal interest in a particular home deal, or provide a commission to people who bring in data on a home that is later successfully syndicated/sold even if the reporter isn't in on the deal. Maybe there are other approaches. Then grow from that segment to more real estate data if the model proves out.
As other comments have stated, it is so much more than a simple technical problem. An entire web of "ethics" rules and well-funded politicians presents a formidable barrier to change. For true blood-boiling outrage, read the very sharp analysis of how the industry is rigged in this report submitted to the FTC a few years ago: http://www.savvybroker.com/Nine-Pillars-Citadel-aRealEstateC...
As technologists, try reading each of the points in this report as network partitions, and design a solution to route around the information block. There are various approaches available now to bypass some of these rules.
Alas, they can rewrite the rules, too, as necessary. For example, in Texas realtors were taking bulk listings from builders and investors for a flat fee of $150 and putting the data in the MLS, but nothing else. Since this was a direct attack on the semi-fixed price of 6% commissions, the Texas Association of Realtors got their politician lackeys to pass a new law, requiring "minimum professional services" from all Texas real estate license holders. This fixed price posting still goes on a bit, but not in bulk anymore. Any technical routing around of the MLS must expect the Borg to counterattack.
The realtor associations even violate their own rules, and/or keep their rules vague, so when the time comes they slap everyone around by giving them days to comply or they are cut off from MLS. (That cuts off an entire brokerage when one agent is out of compliance.) With that kind of career-ending leverage, they are quite effective in keeping the sharecroppers down on the farm, and paid up on their dues.
I have never been a realtor, but I did follow a carefully laid out plan to be licensed and still avoid the Borg, which the realtor association ignored completely. Feel free to contact me if you want to share stories and ideas.
I'm curious to know if Ryan has spoken to any of the local (I live in the same town) agents about the lock-in to see what they think? I don't know a huge amount about the industry, but it has some interesting dynamics.
There are a handful of major franchises (Re/Max, Century 21, CBRE on the commercial side) that folks sign on to. Why are such companies so enticing to agents? Is there a benefit to doing the same thing as every other agent in your region, or is the barrier to standalone entry to the market too high?
I presume that the franchise groups are heavily wedded to the existing MLS stuff, even if they have their own separate listing facilities (certainly the largest Re/Max group here puts all their listings on their own website).
I should talk to one of my neighbours about it, as I have some interesting ideas :)
While I agree with making this information more open, claiming that real estate agents are getting their percentage from just accessing the database is wrong. There is a significant element of service that I have experienced that transcends just having the listing. I have worked with one real estate agent for multiple houses now and gladly pay a percentage to understand the flavor neighborhood, the history of the house, getting the transaction done, helping with the move.
If the information would be free, then I would certainly be ok with paying less, but there is more to the real estate service than just the data.
Redfin is great, but it only solves part of this problem. In its current incarnation it still has to play by many MLS rules and does not open the data past users of its own site.
The biggest brokers in an area will monopolize 60%+ of all the data in a single MLS. When they have control over the listings, they have control over the MLS. If they threaten to leave an MLS, the MLS is worthless, and one company wins.
So, the MLS compromises. It does the bidding of the biggest listing offices and everyone else suffers.
I speak from first hand experience as a vendor in multiple markets. An open mls would be mind blowing and wonderful, bit it's not a matter of setting up a nice website, it's a matter of beating through the bureaucracy.