August 15, 2007 by Nobody Fugazi
Posted in
Everyone has been checking out SLBrowser, so I decided to give it a whirl. With a quick teleport to SLBrowser inworld headquarters, I managed to get one of the HUDs. To the right, you can see the image I got when I did a search for 'Your2ndPlace' by typing ?Your2ndPlace with the HUD attached.
That is kind of cool. It is an effective way to search; indexing items around the virtual world and making them quickly available. There are problems, of course - where simulator resources are limited by other scripts on the server, speed is lower. In fact, as a HUD the SLBrowser is highly dependent on the immediate environment of the avatar. And texture loading may sometimes be slow as well. People all over will find it useful, but may want to find low lag areas to use it.I'd suggest a web interface, actually - and if I'm right, that is probably already in the works.
But understanding how it works leads to 2 pretty big questions.
Is It Legal?
While I don't know the intrinsics of the scripts, everyone knows that uploading textures costs 10 Linden dollars a pop. That isn't cheap, and using the asset server for texture keys to display a texture is only supposed to work if the person who is attempting to use them has the appropriate permissions.
But these images don't belong to SLBrowser. For example, the Your2ndPlace.com logo that appears in the screenshot in this entry belongs to me - so how did they get it? The path of least resistance would be to create a bot which teleports around and grabs textures. SLBrowser then automagically adds a watermark to it and uploads it - and since it is free advertising for businesses, there is unlikely to be a complaint about using copyrighted materials in this way - but if one does not defend one's copyright, the copyright claim becomes diluted. It isn't an exact science; it is a realm where lawyers and courts can debate back and forth. It is a stumbling block, perhaps, and one which is better left for lawyers to discuss, especially with variations in copyright law around the world. Plus, the images I saw had an SLBrowser watermark across them, which is actually an alteration of my recognizable work. [w:Fair Use]? Maybe. As long as the user of the work doesn't benefit financially, it may very well be Fair Use. I'm just someone with some copyright and trademark disagreements in my past (and probably my future), and am aware of these things. When Benjamin is around, he'll hopefully drop a comment or write an entry.
For my part, I like it. I respect that it was done, and as such I'm publicly giving SLBrowser.com the Rights to use my copyrighted materials for non-commercial use in their HUD for the strict purpose of showing search results. So they are clear with me, as far as I am concerned, and I have protected myself as well. Maybe it is overkill, but copyright is my bread and butter. I take it seriously, it puts horseradish on my roast beef. Heck, I got offered 100 British Pounds for a photo I had on Flickr today, but Linden Lab press beat me to the shot of a Ginko ATM for the Economist. Grr.
Still, that is probably something which should have been addressed much earlier on - though begging for forgiveness is sometimes much easier than asking for permission. I know I've been guilty of begging for forgiveness a few times myself, especially when pushing the envelope. Without pushing the envelope, nothing gets done. My stance is pretty clear - sure, use my work, but if you collect money, well, we may have to rethink this whole thing.
Hats off to Diva Canto and Felix Wakmann for pushing this envelope, and I hope that if Linden Lab runs with the idea you get something out of it - well, more than fictional currency.
Will A New SecondLife Search Make It Obsolete?
Probably. Linden Lab has all this information already on their servers. They could easily query against objects for sale, textures used on the objects and many other factors. Where they are lacking, as most of us are painfully aware, is infrastructure. The problem with the HUD is that it is dependent on available resources on the simulator; a web interface would be less affected - and since the HUD is a listening device, the HUD also generates lag.
If Linden Lab decided to do this, though, they wouldn't need to pull the data using a bot - they already have it!
Overall
Yeah, I'll use it when I need to search for things quickly - but I do have to admit that some of the most interesting things I found were through wacky search results. If this is a new era, and I believe it is, I will miss those wacky search results. Seriously.
I do have questions about how rankings are done, since the rankings themselves will affect what people purchase - which will cause gaming of this system as well. And further, I wonder whether the use of the HUD is being recorded with avatar names; if so there may be privacy issues involved. Creating a database of intentions is all well and good, but how the digital dossiers on avatars are used is always in question.
In all, I'm glad to see SLBrowser - but to be balanced, it brings up some serious questions as well - and to ignore those questions only means that they become delayed.
- Nobody Fugazi's blog
- Add new comment
- 2131 reads
privacy
hey, you know these slbrowser bots are crawling through people's houses and listing what's in people's houses.
And, to boot, providing the searcher with a link to directly tp out of the middle of nowhere into people's living rooms, day or night.
robots.txt
I remember when Alta Vista happened to the Web. I remember being somewhat shocked to see how easy it was to find my web pages, when the practice of the time was to send those URLs by email, word-of-mouth, and by listing them on the portals that had started popping up just the the year before. I remember having felt somewhat uncomfortable with the idea, at first. OMG!!! People could find me just by searching for my name! That was a total loss of control over disclosure of my Web content. The advantages soon proved to outweigh the disadvantages by several orders of magnitude, so I got over it pretty fast, and so did everyone else.
By that time, a group of people had devised the robots.txt protocol, which may or may not be honored by web crawlers, but at least it's a nice thing to have. No such thing in SL yet. I hope our work with SLBrowser will prompt the creation of a similar protocol for SL sims. If any techno-weenies out there would like to start talking about it, let's do it like the robottxt group did -- voluntarily, by consensus, and without any standards body. It would be nice if this was made at the server level, but it really doesn't have to be there, it can be at the applications level -- for example, an object in the parcel called "NoBots Please." Our crawlers will get the message; hopefully other bots also will.
(In case you're wondering, the way the servers work, it's not enough to place our bots in the "ban" list of the parcels. The bots can stand outside the parcels and "see" some of the objects inside. So we really need a protocol here that means "even if you see the objects in this parcel, please ignore them.")
an opt out list.
Basically, an opt out list would give people a general way in which the bot could validate against. If someone is on the list, their content is simply not indexed. Since the bots are client based, that shouldn't be an issue. a central server of avatar names and/or keys which don't wish to be indexed.
I don't think many folks would opt out, but - it gives them a chance to do so. A simple txt file on the web would suffice.
I remember robots.txt, when it came out and the effect it had. Things have changed a bit since then. :-)
yes, but *how*?
Opt-out is good, and everybody understands the general concept. The question is: what's the best way of doing that opt-out mechanism in SL? We (SLBrowser) could provide and maintain a list of opt-outs. That will work for *us*, but won't solve the problem of "boting" in general. Quite honestly, we're not interested in quick-fixes. On the Web, that is solved in general with robots.txt -- a file that all bots, independent of who writes them, should honor.
The other issue is what's the granularity of opting-out. Should it be account-based, like you suggest? (i.e. all objects owned by a certain avatar) Or parcel-based? (i.e. all objects in a parcel) Or both? What will the language look like? What are the best identifiers, given that keys change when objects are deleted and rezzed again? Where should the interface to this be? On the Web (in what server?)?, inworld? on the sim server protocol? Etc. etc.
Anyway, this issue is impossible to avoid, as more and more people start writing their own clients to interface with the SL sims. We'd like to play a part in defining that protocol, because we don't want to step on anyone's privates (metaphorically and literary). Plus, the less unwanted (noise) objects we index, the better search will be. We really don't want to list stuff that is accidentally on sale in someone's bedroom.
I'm not sure anyone is listening, though. The topic of bots seems to provoke strong negative reactions from very vocal people who scare everyone else, when it is, in fact, a technical issue that needs to have a technical approach as SL opens up. One of the consequences of opening up is precisely the ability for people to write their own clients. Let's move on, shall we? The next question is: what's the best mechanism to deal with bots' potential for intrusion, invasion of privacy, and disruption? The answer for this will necessarily be part technical and part legal. Neither of it is done for SL, yet.
Granularity, et al
Well, it certainly would help if they open the server side.
Assuming that they don't, it would start off as avatar based. Why? Because avatar based accounts for parcel ownership changes. Either you want 'em in, or you don't. Then there is the issue of vendors (which at this point, personally, I think shouldn't be indexed as a matter of practicality. Vendors could be easily used to game the system).
Parcel based could be an option, but that would require real time information from LL's databases - who owns what, etc. Now, that information is already out there - I know, for example, that landbot owners had already mapped the entire grid months ago. As a matter of technology, you could have those zipping around and collecting the same information the hard way. The easiest way would be for LL to provide an API call that simply ran a query on who owns what.
But then if LL were on top of things, they could just set parcels to 'indexable' or not based on a checkbox.
Bots -
Unscrupulous landbot land dealers have been ripping people off for thousands of dollars a pop, and it has a lot to do with observed network latency, as well as other things. If you google 'Landbaron Merlin', you'll find a few things I wrote. In one week, that particular character pulled in $10,000 US through exploitation of systems - and LL, of course, can't acknowledge that. One of the reasons that I stopped dealing in mainland.
So yes, that is a real issue, and people tend to get emotional when they lose real money (not the 'fictional currency' mumbo jumbo phrase that keeps getting batted around like a seal between Orca)
If LL were to actually have policy evolve with the technology, well - that would be brilliant. If your bot does bad stuff, you get banned. The crux of the issue is that LL is unwilling to define 'bad stuff'.
Technology doesn't exist in a vacuum. The issue of bot usage is not just a matter of technology - it is a matter of policy, and LL desperately needs the intestinal fortitude to deal with some of the issues. It took them forever to ban gambling, but a few people versed in international issues knew it had to happen. They dragged it out, it was painful. LL need to learn how to amputate quickly. It hurts, but it heals a hell of a lot faster.
You also have to consider that many SL users don't even use office software. Spreadsheets may be new to them. Heck, I never used the Gimp seriously before I got to SL. So expecting these people - regular people, instead of the hardcore gamers seen in other virtual worlds, macroing their hearts out - to adapt quickly to technology change when the first use of it was negative... well...
That is sort of like the first atomic bomb being used, then talking about nuclear energy within a few days. In reality, lots of policy issues happened to deal with nuclear energy - and we still have issues. I think Feynman would have a field day with this stuff.
And that links directly to technology use and adaptation, as well as...
Standards -
The core issue of customized clients is exactly as you say - standardization. I think that there is a lot of room for that, but I also believe that true standardization approaches cannot occur until Linden Lab opens up a bit more. And then, I think that true open standards have to evolve based on more than one virtual world for them to be useful.
I haven't fired up my compiler on the client since it first came out for similar reasons. I peruse the code off and on, but I just don't have the time to fiddle with code that isn't standardized - and time spent finding contracts is much more well spent. At least writing brings in some income. :-)
Frankly, with my confidence in the Linden economy at this point, I don't want too much investment (time, energy, money) into something which may not last another evolution of Moore's Law. Right now, my focus is on my refrigerator. That should change within the next 12-18 months, and from there I'll see what I see. For all I know, SL may have different owners by then. :-)
I've been following other virtual world development stuff - SL does have a lot of advantages in some regards, but it comes with severe disadvantages and a lot to grandfather in.
I expect I meandered here, but I'm tired. Hidden in here are some key points which I have actually been stewing since I hopped inworld to Asheron's Call and got ticked off that the Life Mages were killing all the stuff through walls. Bastards.
In all - yes, yes, and YES. I agree. But as Feynman said somewhere ("Meaning of it all: Thoughts of a Citizen Scientist" ?) - scientists have to be guided by society to do the right research... and this is the case with any technology. There is no question that the SLBrowser is much better than what we have now, but someone's gonna gripe, and then when the rest of the community catches up... well, then they'll say what is on their mind. But it takes time for the community to catch up. What I tried to do with the review was step in to some of the issues quickly so at least informed discussion can be had.
But if you look at this thread... well, we're the only ones talking. And I think I've meandered enough... (eyes crossed) - but I would suggest consolidate technology/docs now, give it some time, and there will be more discussion. If this is the spiral model, you gotta let the users look before you hit the next evolution. Yeah, users can be a pain, but as all developers know - we need 'em.
the father figure
I just don't buy into the argument that Linden Lab should be the regulatory entity, who happens to not be doing a good job at it. Yes, SL has lots of oddities from an architectural perspective, and it creates policy problems that didn't exist (as such) before but that can easily be mapped to problems created by older technologies. More oddly, though, Linden Lab is in a strange situation of being both the developer of this general-purpose publishing & communication technology and the only host in town. That causes a lot of confusion.
The only reason why I'm investing my time here is because Second Life, unlike all other virtual worlds, has the potential of becoming like the Web -- and that means different organizations being able to run their own sim servers, just like they run their own web servers. In that scenario, it's pretty obvious that Linden Lab cannot (and should not) be the regulatory agent. The stuff they did with casinos is simply PR aspirin. Casinos will pop up again in sim servers run from jurisdictions that allow them. And bots will be a perfectly normal reality (they already are). Should Linden Lab regulate all of that? heck, no! There needs to be a W3C of sorts that defines a strong technical protocol, along with users taking legal action against fraud when it occurs. If I were LL, I would be speeding up the Open Source release of the server asap! -- that will clarify the confusion of responsibility that exists today and that LL is addressing with a sequence of aspirins.
Also, in the scenario that I'm describing (which is the one most companies are interested in), search and services like it cannot be done in a centralized manner like you were suggesting in a previous post.
Also...
Reference: this related to concerns about bots.
Another concern, which relates to the discussion - where a policy related to 16m plots (or lack of it) could easily be affected adversely by bots.
And who regulates the bots?
You see, it is not so simple. There are many factors involved in this - some that the community does know about, others which the community does not know about...
It is not strange that the first bots have been used to make money. After all, there are no financial rewards for working on the code otherwise.
And where code becomes de facto law so much of the time, it is important to have policy in place to handle such things.
No, No...
You may not buy into it, but in the points I made there is really only one entity which can do something. Maybe something like that hasn't affected you personally yet. When you can figure out how a woman in Germany can get $1,500 US from a John Doe landbot owner, or a woman in Italy $2,000 US from the SAME John Doe, and you can say that it is accessible and therefore enforceable - you have something (and these are real issues). But you really can't, not without involving Linden Lab in the lawsuit. Some say that this may happen soon. Linden Lab is in the middle, even if it tries to sidestep it with the ToS.
If Linden Lab provided the community more tools to self-regulate, well... to borrow your metaphor, Linden Lab is saying, "I'm not your daddy, I'm not your daddy" but the reality is that people don't want a 'daddy'. They want solutions. They want tools to create those solutions.
And lets be frank. Linden Lab created its own situation.
Now, you're talking about legal actions an such... and at internet governance levels, that still hasn't been sorted out. To expect it to be sorted out here without internet governance issues even on a concrete path... well, draw your own conclusion.
I get where you're coming from - trust me on that. But on a different level, policy has to be in place. It could be Linden Lab policy, or we could wait around for internet policy, or Linden Lab could provide for tools which do allow for self-regulation. We're all going to be up to our eyeballs in lawyers at the rate we're going.
For the record - bots are a 'normal reality' for those who have them, policy is needed regarding bots (be it by Linden Lab, or enforceable by community - but if community has no teeth with the tools provided, well...).
If Linden Lab every OPENED its server source, the community would probably have already stepped in. But they keep adding 'features'.
Saying that the casino ban was PR aspirin is something I do not understand. Credit Card processing companies simply don't support the majority of those transactions, and that is policy. The casino PR ban was a reaction to policy by organizations Linden Lab does listen to - people who impact their bottom line.
Now, you're talking about open servers, and search and services... Don't you think if they opened the server someone could write a SQL query and get the same results without a bot? :-)
Interesting discussion. More people should be involved in discussion like this - when that happens, it will be a good thing. However, there are just a few of us talking at these levels, and quite frankly - that is not enough.
And that does mean involving people in discussion... I wonder if anyone else will bite on this discussion. Let's see.
Designing Second Life Robot Control Protocol
Here are my requirements for a "robots control" system:
This will be a voluntary system. The robot writers will have to decide to obey the policy of the land they visit. The SLBrowsers will obey the policy when it is developed, and I think most other crawlers will too -- the public backlash from not coorperating will be too great. In other words, a voluntary system will probably work well in practice.
How to meet these requirements? I think it is pretty simple. We create a simple protocol on a set channel. When a robot enters a Sim it can make a request on the "robot controol channel" and listen for responses. Land owners just need a simple device that listens for requests on the channel, and then sends back instructions on whether to crawl that parcel. The response could also indicate options for what kind of crawling is allowed.
In practice, all a land-owner would have to do is get a simple, free, 1-prim "crawler ban" object/script and put in on their property.
All we need to do is define this protocol. we don't need to wait for the Lindens, who have other fires to put out. I'll mock someting up. If anyone has any comments, I'd love to hear them.
I also want to point out that "doing a SQL query to get the info" from a server is semantically the same as "using a bot to get the info". The privacy issues will be the same in either case. Its not the bots that are the issue, its the information they collect.
NOTE: I tried to read up on the people stealing money using land search. I could not figure out what is really going on. I think it had something to do with shifting prices while people are in the middle of buying. This is a classic "database transaction" problem, and if you can change the price between when a person agrees to buy and the transaction takes place -- well, that is just a particularly nasty bug in SL. But maybe i'm missing something here.
the hosts and the techmakers
I agree that LL is risking legal troubles ahead. Hosting services risk it, if the service they provide proves to be vulnerable to scams due to their own actions -- in this case, the action being the release of the client, which made it trivial for hackers to write all sorts of griefing clients. But we all know that LL has the best of all possible intentions with that action and, in fact, that action -- plus the promise of the future release of the server-side code -- is what attracts hoards of people and companies into this.
So, they're in a tough spot. Hosting services like AOL, Yahoo, facebook, ... and, yes, SL-as-host must act as micro-governments ensuring that their citizens are safe. If LL was serious about being a 3D hosting service it would keep its code under wraps, like all other virtual world companies do, and impose rules and conditions about what can and can't happen on their servers.
Luckily for all, they aim at being technology paradigm-shifters. And that's somewhat in conflict with their other role. I think we all see the result of their internal bipolarization.
A technical clarification. You say:
>Now, you're talking about open servers, and search and services...
>Don't you think if they opened the server someone could write a SQL
>query and get the same results without a bot? :-)
That's not how the servers work right now -- it could be but it's not, and I can explain why off-line. It's unlikely that sim servers will ever provide that kind of service, because of performance issues.
Web servers also don't work like that -- they don't give out the complete listing of the pages they serve. (They could, but they don't)
Thanks for the feedback
Thanks for a thoughtful discussion of our product. Unlike Electric Sheep, we are just two SL residents who build things in our spare time. We don't claim to have all this worked out yet.
Some corrections first: we do not copy your image and upload it. And SL does not have any security around textures. Any texture that you can see in-world has a Globally Unique ID (GUID) which is given to the SL client in order to render the texture. As far as I can tell, once you have the GUID, the texture can be aplied to any object in world, no matter who it is owned by or what the permissions are. I'm not sure whether this is correct, but it appears to be the current behaivor of SL.
We added the "watermark" to the images in response to user feedback. Some textures were being shown that people did not want to be easily captured (via a screen shot). The intent of the watermark was to change the image to make it non-usable. We don't want to imply any ownerhsip by SLBrowser. We could just put random scribbles on it instead.
Our display of these images is similar to Google's display of images. Google makes money on their search. I wish I understood the copyright consequences of all this.. but it all new. One idea I had was that you could put a copyright notice on the bottom of your image, but no display that part of the image in-world. SLBrowser displays the full image, so the copyright would appear.
I'm not sure the Lindens can do better than us. I posted this to SLInsider: In talking with a Linden recently I realized that they couldn't do this service any better than we have, even with their inside knowledge: they don't own all the objects, or have much more knowledge about them than we do. I think any search service has to be based on crawlers that act just like ordinary avatars and see what they can see. If the Lindens tried to go in behind the scenes, they would be violating the intellectual property rights of the owners of the objects.
As for collecting private data, Diva Canto is working on anonymizing our aggregate search/hit data. This will be used to improve ranking, and we don't need avatar identities to achive that goal.
I don't claim to have all the answers.. but we are happy to be part of the dialog abot how the metaverse will work.
GUID
Yeah, I got the GUID stuff on my own today. There's no real way to protect them from being copied, since they appear on everyone's screen (part of the misunderstanding many people had on the Copybot scandal).
As far as the Lindens - well, their servers are where the GUIDs are, where the objects are kept track of on parcels around the grid, where prices for objects are maintained... if they haven't put 2 and 2 together yet on this, then they need some better SQL folk running around. Everything is in tables; it is a simple matter of pulling data from the tables. That is all you are really doing, only from the outside - and using simulator resources to run the script...
Frankly, I don't think that their problem is that they don't know how - it is that they don't have sufficient manpower to throw at the problem.
And as far as your discussions with Linden Lab, count yourself fortunate. When writing my first O'Reilly Shortcut, after months I got handed off to LewisPR. When people tell me that they have had conversations with the Lindens, you must forgive me - I roll my eyes. Even that you had a discussion boggles my mind. Maybe my questions were too hard and in sensitive spots, or maybe I don't live near enough to California or MAYBE they just plain don't like me. Out of everything around such clandestine discussions, I see where Prok comes up with her FIC theory - it is consistently peculiar how they talk to some and not others.
That said - great work. :-)
Very cool & balanced review
Good point about the watermark, I had not noticed it was there! I would have said that use of those textures is much like having your picture show up in a google search, but google doesn't add their "stamp" to your image. I'd be curious to see why Felix chose to do it this way...
For now, I'll join you in being respectful for the effort Diva and Felix put into this product. They also commented on my post at SL Insider (http://www.secondlifeinsider.com/2007/08/15/sl-browser-in-the-run-for-be...) and listed their bots so that people can choose to ban them from their land. This night come in handy in giving people a way to protect themselves until a better method is coded.
-Thea
>> This night come in handy
>> This night come in handy in giving people a way to protect themselves until a better method is coded.
I think this emphasis is wrong. The bots are crawling through people's houses. In RL, if Google said, oh just let us know if you don't want your bank account crawled -- they wouldn't get away with it for a second.
This issue should have been a show-stopper in the development of this product until it was resolved.
Chaz Longstaff
Robot control and a bug
I'm not quite sure how to respond to this. We are not crawling your bank account, so I'm don't think the analogy is fair. There are lots of analogies that could be applied, like searching a MySpace page, or taking a picture of your swimming pool from a satellite and putting it up on Google Earth. What metaphor is the correct one?
We want to engage in a constructive dialog about these things, while we experiment. I'm sorry if you would prefer that we had everything perfect before we started getting users -- but we think that we cannot make it right until we get user feedback.
Our intent is to only list objects that are "For Sale". Some users recently reported that we incorrectly identify some objects are for sale, when in fact they cannot be purchased. We are working on a fix and hope to have it rolled out soon (perhaps next week).
Chaz does have a point...
Maybe he came across strong and used the wrong metaphor, but there is an issue of privacy (which, oddly, I did not bring up) that does need to be looked at seriously - I know you are doing it, but he makes a valid point: The genie is out of the bottle.
This is not to say that the bots for SLBrowser are the only ones wandering around like that - I'm sure that they are not - but SLBrowser does have the misfortune of being the one who made the problem visible.
Engaging in 'constructive criticism' at this point does, for all intents and purposes, seem like a discussion at gunpoint. Some people take their privacy in SL somewhat seriously, and the point that it probably should have been factored in before the release, sans the bank account metaphor, is valid.
Dealing with that concern is important, especially understanding that the people concerned with their lack of privacy may have already lost it.
That is my constructive criticism.
We hear you
We are working on several adjustments to the way our search works based on user feedback. I know that we have forced the issue and gotten people's attention. Seeing the real system in action gives us a much better basis for discusison. On the other hand, if you are really concerned about your privacy, perhaps you shouldn't put personal information into publicly visible places? Anyone can fly by and get information on land. I don't put my name on my house in RL or SL, for exactly this reason.
Again, we hear you and are making changes. But keep in mind that SL is not a finished metaverse but a work in progress. If we were perfect (or a bunch smarter) we would have done this better; but we are not and the cat is out of the bag, so we have to work through it as best we can. At least we are being public about what we are doing, instead of collecting data privately and using it in ways that you don't even know about. That should give us all a moment's pause for reflection.
Fixes applied
Our latest crawler now has two fixes based on user feedback:
1) if you do not want your parcel or its products to be listed in SLBrowser search, then go to About Land, Options tab and uncheck the "Show in Search > Places" checkbox AND also set the category to "Residential". The dropdown has meaning even if the checkbox is not set. SLBrowser will not list any parcel or products from parcels marked in this way. The robot may still visit your parcel, but will not collect any information. If you land is listed for Sale, then it will be included in the parcel list. We update our index every week, so it may take a week for the change to be applied.
2) We fixed a bug where some objects were being listed even though they were not actually for sale (these objects where already purchased).