Russell Beattie Notebook

The XAO of Pooh: XML Access Objects as a New Pattern for Web Development

Posted on Thursday, July 31, 2003 04:29 PM

Instead of actually producing anything over the past few months, I've been instead trying to find the perfect system for producing web content. This is, of course, my way to procrastinate. I get to play with a bunch of different technologies, learn a bunch of stuff, write a bunch of code and at the end I actually don't have anything accomplished. However, finally, I think I may have stumbled upon the system I've been looking for and I'll explain it here.

First you have to understand my opinions about web development. Doing web dev is not brain surgery. What you're doing is simply reading data from a database and formatting to present it to a browser, maybe taking some data given to you sometimes and popping it back in the db. That's it. Everything else is just abstractions from this basic data read/write paradigm. And that's the stuff that drives me crazy: abstractions.

Russell's #1 rule for app development: For every layer that you put in between you and your data you better have a damn good reason.

However, that said, there are some good reasons out there. Slapping all your code into .jsp pages is definitely not great from a maintenance standpoint at all. My website has survived some decent size blasts from Slashdot, Wired News and averages over 750,000 hits a month so it's not a bad way of doing things from a performance standpoint (as many people on the current MVC bandwagons would lead you to believe) but JSP can quickly turn into spaghetti code. So there's one reason for putting a layer on your website: Separating out your presentation from your logic.

For this, I decided I'm just using Struts. Why? Because it has been blessed by Sun and it has an incredible amount of docs and support. There's like, what, 5 books, endless numbers of websites and constant work being done to improve it. I'm not Struts biggest fan, but I got it to work like I wanted it to with some tweaking so I'm sticking with it. I looked at developing my own simplified MVC, at JPublish, Cocoon and various Python systems, but decided in the end that Struts did what I needed it to do and for the most part got out of my way, so I went that way. There's a lot to be said for standards and Struts is becoming the MVC standard. (Don't jaw at me about WebWork. I looked at it and it just had *too many abstractions*. Bad.)

In this process I took a serious and hard look at Cocoon. I decided that it was too heavy. Too much of big black box who's way of functioning were a complete mystery to me. And waaaay too much use of XSL. However, I *like* Cocoon's way of thinking a lot. I learned a ton by messing with it and came away with two must haves for any future web dev I do.

First, the URL that's sent to the web server has to be completely separated from the logic and data that's returned. Cocoon provides a great mapping layer where you can define the URLs you want to respond to using RegEx expressions and it works great. This allows you to present your url like http://www.russellbeattie.com/notebook/20030728 and have it correspond to a dynamic query. There's no reason for anyone out on the web to know you're using Java or Python or who knows what because of the URL you send. You should have complete control over the URL.

Cocoon is a lot more powerful that Struts in this respect and uses a lot of server-side magic to make this happen. At first I tried to get Struts to work the same way by just hacking a servlet mapping with /* into the web.xml of my app and then from there handle everything in a custom servlet, but it's a lot of work and more processing - doing it this way, you're responsible for all the .jpgs and .gifs and .css and every other static file as well. The *other* reason is that you can't use .jsp pages mapping /* because eventually when you do a Forward to the .jsp page (after you've done your logic on the back end) the servlet container balks because of the cyclical reference. In other words .jsp matches /* as well... There are some work arounds for Tomcat - you can define a /*.jsp first and point it at Jasper, however it's not a portable solution and again you have to deal with all the static documents so I decided not to go that route. Instead I decided that I would map *.html and *.xml to my custom servlet instead. This servlet simply throws the original path into request object attribute and the forwards the request off to a Struts action. Like this:

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;


public class URLProcessorServlet extends HttpServlet{

    public void doGet(HttpServletRequest request, HttpServletResponse response)
               throws IOException, ServletException
    {
        execute(request, response);
    }

    public void doPost(HttpServletRequest request, HttpServletResponse response)
                throws IOException, ServletException
    {
        execute(request, response);
    }

    public void execute(HttpServletRequest request, HttpServletResponse response)
    	throws IOException, ServletException
	{

		request.setAttribute("originalPath",request.getServletPath());
		String redirectUrl = "/do/URLProcessor";

        RequestDispatcher requestdispatcher = request.getRequestDispatcher(redirectUrl);
        requestdispatcher.forward(request, response);

	}

}

Then the Struts action takes a look at the URL and processes it in a big if statement fowarding the request to specific actions that do the real work before forwarding to the jsp pages. I use a standard header on all the pages, so to make sure that someone doesn't call a .jsp page directly, I simply add this bit of JSTL at the top of the .jsp page:

<c:if test="${empty originalPath}">
	<c:redirect url="/index.html" />
</c:if>

Do you see how that works? Basically, if the .jsp page didn't get called through the URLProcessor servlet (which is actually badly named because all it does is forward the request on... but I wanted it to match up with the Action for clarity) then the "originalPath" attribute isn't set, and the page gets sent back to the front page.

So that's the first thing I got from Cocoon - the desire to have full control over the URIs that my server sends out. Having to have *.html at the end is a bit of a hack and if you read TBL, he says it's not a future proof URI, it's still standard enough that most people won't even look at the URL.

The second thing I got from Cocoon is the idea that XML is *the* only way to use data. The first thing you learn in Cocoon is that you need to start with XML before you can do anything else. Step one in the Cocoon presentation process is done by using a variety of "generators" which produce valid XML. Query a DB? It has to come out as XML. Files, URLs, whatever, before you can start manipulating it with Cocoon it has to start as XML.

This is a *fantastic* way of thinking. At first I fought the concept because, well, XML is a pain in the ass. Producing valid XML from random data sources is a real bitch. But if you want to eventually produce different types of XML like WML, XHTML, XHTML-MP, SVG, etc. the only real way is to start with valid XML and go from there. With XML it's really garbage-in, garbage out so you've got to start clean.

The idea is that you have to really have bought into the idea of XML as a flexible data transport. If you think XML is just pure crap, then there's no reason to continue reading because you'll think that everything from this point on is a pointless exercise. But if you get the idea that once data is within XML that it becomes super portable and transformable and incredibly useful and worth the inconveniences, please continue.

So as soon as I decided not to use Cocoon, I still wanted to keep this style of programming because I think it's just makes incredible sense. It passes my layer test: Yes, I'm putting a layer of XML processing in between me and the database, but the benefits of this extra layer (which I'll explain shortly) justify any drawbacks.

Happily, once I moved back to just "plain" development with .jsps, I discovered the joys of development with JSTL. It's *really* well done. Those of you with long memories might remember me almost a year ago on this blog bitching about how JSTL is horrible and ugly and a misuse of tags, but I was completely wrong. That was a reactionary opinion from someone who spent years with embedded Java in their JSP. The JSTL is done very well and is quick and easy to produce clean JSP pages. I really look forward to JSP 2.0 when the Expression Language is used throughout the page, but for now using just the tags is fine. It's still an abstraction - all the tags are doing is producing a servlet at the end, but it's an abstraction worth using as well (as long as you don't bump into the dreaded 64k class limit. Ugh.)

The absolute *best* thing about JSTL is the XML processing tags. They are INCREDIBLY powerful! I can't believe how cool they are. Using a simple import tag, you can grab your XML from anywhere (file or http request) and then use the other XML tags to loop through the XML and present it to your users. I don't really like XSLT - it's okay, but it's difficult for me to use. I mean, I do like how XSL is separate from a server implementation - an XSL stylesheet you write can be used by any XSL processor, but other than its portability, XSLT is just a bitch to use. Errors are cryptic and the results can be many times incredibly mysterious. Add any sort of complexity to the transform and the XSL page can be *HUGE* as well.

The one thing I like about XSL is its use of XPath. And happily, the JSTL uses it as well! AWESOME! So now you can use the power of loops and if logic on your page like you would normally do, but instead of doing it with SQL data, you're doing it with XML data queried via XPath. It really works well and the development time (when compared with XSL) is insanely quick. But hey, if you want to use XSL, JSTL supports using XSL stylesheets just as easily, so you don't have to pick or choose. Some things will naturally be easier to do via logical loops and ifs, other stuff will be more straight forward transformations.

So now I've come really, really close to having everything I need to quickly and easily produce modern web content via XML, but without much excess baggage. I have control over the URLs, separation of logic and presentation and presentation based on XML manipulation or transformation. The only thing left is the XML itself. And that's where XML Access Objects come in (finally). It's a revelation that just recently came to me and I'm still working out the issues, but hopefully you'll see the core of the idea and why its so cool.

You can think of XAOs as Data Access Objects for XML. They separate the application from the source of the data (which can be a database, an xml page, or whatever), but instead of returning some sort of Map or Collection of Beans like DAOs do, these classes only produce XML.

Here's the XAO interface:

public interface XAO {
	
	public String getXML();
	
	public String getXML(String xml) throws XAOException;
	
	public void setXML(String xml) throws XAOException;
	
	public String getSchema();
	
	public String getParamSchema();

}

What this does is create an standard interface for dealing with XML data in your application, but without specifying the implementation. Don't be mistaken by the first glance - XAOs aren't an object representation of XML or anything like that, they are simply XML generators. Once you've got that XML in your hands (generated from any source) your options are incredibly broad.

In fact, there's so many cool things you can do with XAOs I almost don't know where to start. XAOs aren't simply interfaces to XML data, they're also XML encapsulation of *any* data. Though the name is related to Data Access Objects, in use they're more like "XML Java Beans". Instead of encapsulating your data in a million varied JavaBeans, Collections and Maps, you instead use XML and reap a ton of benefits. (Maybe I should call them XEOs).

The first rule of XAOs is to keep them simple. The second is that that they produce only XML Strings. The third is that they only consume XML Strings. That's it - anything more is too complex and too abstracted. (Well, I say "only", but I've implemented them as an Interface, it's really just a rule that applies to when you're working within the framework of XAO.)

Here's how I see the interface working. A getXML() query always returns a default XML document. This can be dynamically generated from a database lookup, a local file store, an http request to another server, a XQuery to an XMLDB, or if the interface is set on top of a regular JavaBean or even an EJB, it can return the contents of the bean itself as XML as well.

But what does that XML look like? It can be any XML data you want, but you need to be able to communicate what its supposed to look like for other classes to use the XAO, right? So that's the job for the getSchema() method which will return a valid XML Schema document which describes the XML returned from the getXML() method in detail. (If you haven't used XML Schema yet, well, neither have I... but it's the best way to mimic JavaBean-level of detail in the data returned). I honestly haven't gotten much into this yet and have added it in for completeness - but I think it's important. Another option is to return a URI or a DTD instead, but that breaks rule #2. But since I just made the rules up, this may not be a problem. :-)

This same schema is also used when you want to push data to the XAO as well with the setXML(String xml) method. The XAO is expecting a document that it knows what to do with, so it has to be the same as the document that's returned with the getXML() methods.

There are obviously many, many times when a default data lookup isn't going to be enough, and that's where the getXML(String xml) method comes in. This will allow you to pass an XML document of parameters to the getXML() method to focus your "query". What does the XML that's passed to the XAO need to look like? Well that's where the getParamSchema() method comes in. This method returns a definition of the XML that the XAO knows how to parse in the getXML(String xml) method.

Now if all this schema stuff makes this seems complicated, it's because it is. The caveat I've decided on to make this process the most flexible is that if the schema methods return null, then *any* xml document can be returned or set. If you send something that the XAO doesn't like, it'll just throw a XAOException which you'll need to deal with programmatically. But by implementing the XAOs using the schema methods, you can then programmatically parse and validate the XML. For many applications this level of detail might be required, otherwise it's just a data free-for-all.

So here's some examples of how using XAOs can be insanely powerful. Here's what I'm doing for the next rev of the software that runs this site. First, I create a new XAO class called PostXAO which will allow me to grab my weblog posts from a MySQL db. Here's what the getXML() looks like (for example):

public String getXML(){
	
	Connection conn = null;
    try {
        Context ctx=new InitialContext();
        DataSource ds=(DataSource)ctx.lookup("jdbc/RussellBeattiePooledDS");
        conn=ds.getConnection();
    } catch (Exception ex) {
        ex.printStackTrace();
    }

    StringBuffer xmlBuf = new StringBuffer("");
    String sql = "select * from miniblog where parentId = 0 order by created desc";

    Statement s = conn.createStatement();
    ResultSet rs = s.executeQuery(sql);
	
	xmlBuf.append("document\n");

    int count = 0;
    while(rs.next()){
        count++;
        if(count > 20){
            break;
        }
    	xmlBuf.append("\n");
    	xmlBuf.append("" + rs.getString("id") + "\n");

        String title = rs.getString("title");
        String content = rs.getString("content");

        ByteArrayOutputStream sout = new ByteArrayOutputStream();
        org.w3c.dom.Document doc = tidy.parseDOM(new StringBufferInputStream(content), sout);
        String contentClean = sout.toString();
        contentClean = contentClean.replaceAll("&","&");
        contentClean = contentClean.replaceAll("&","&");

        xmlBuf.append("\n");
        xmlBuf.append("" + contentClean + "\n");
        xmlBuf.append("" + rs.getString("created") + "\n");
    	xmlBuf.append("\n");

    }
	
	xmlBuf.append("\n");
    
    try{

        rs.close();
        s.close();
        conn.close();

    }catch(Exception e){

    }

	return xmlBuf.toString();

}

<document>
		<entry>
			<id>1003490</id>
			<title>Test</title>
			<content><![CDATA[This is testcontent]]></content>
			<created>2003-07-03 04:08:24.0</created>
		</entry>
</document>

I think it'd be better to use one of the XML libraries like JDOM, DOM4J or even Sun's stuff for producing the XML, but many times it's just easier to hand draw it. Now, here's how a XAO is used to grab the XML in a Struts Action before passing it to the display:

public class IndexAction extends Action {
	
  public ActionForward execute(ActionMapping mapping, 
  								ActionForm form, 
  								HttpServletRequest request, 
  								HttpServletResponse response)
	throws IOException, ServletException {
		
	PostXAO postXAO = new PostXAO();
	
	String xml = postXAO.getXML();
	
	request.setAttribute("xml",xml);
	
	String forward = "index";
	
	return (mapping.findForward(forward));
}

<x:parse xml="${xml}" var="mainXML"/>

<x:forEach select="$mainXML/document/entry">
	<fmt:parseDate pattern="yyyy-MM-dd HH:mm:ss.S" var="created"><x:out select="created"/></fmt:parseDate>
 	<c:if test="${empty latestDate}">
 		<c:set var="latestDate" value="${created}"/>
 	</c:if>
 	 <div class="post">
 	 	<a name="<c:catch><fmt:formatDate value="${created}" pattern="HHmmss"/></c:catch>"/>
 	 	<h2><x:out select="title" escapeXml="false"/></h2>
 	 	<h3><c:catch><fmt:formatDate value="${created}" pattern="EEEE, MMMM dd, yyyy  h:mm a"/></c:catch></h3>
 	 	<x:out select="content" escapeXml="false"/>
 	 	<p class="postlinks">
			<a href="<c:catch><fmt:formatDate value="${created}" pattern="yyyyMMdd'.html#'HHmmss"/></c:catch>">Permalink</a> |
			<a href="<c:catch><fmt:formatDate value="${created}" pattern="yyyyMMdd-HHmmss'.html'"/></c:catch>">Comments [<x:out select="count"/>]</a>
		</p>
 	 </div>
 	 <p />
</x:forEach>

If, however, I wanted to produce an RSS document, I could write some more JSTL and create something fun, but instead I'd rather just use good old XSLT. In that case, I could just write instead:

<c:import url="/xsl/rss.xsl" var="xslt"/>

<x:transform xml="${xml}" xslt="${xslt}" />

Now this is all well and good, but it only shows a bit of what you can do with XAOs. Now that you've got the basics, you can start adding some neat things. First, let's imagine that you *hate* Struts and think it's a bloated nightmare of an app, and you just want to use XAOs from within your JSP pages - but still don't want to mess with taglet code. Well, here's a custom XAO tag that I'm working on now. It would instantiate a new XAO based on the XAO named and grab the XML based on the interface. Here's what I think they look like now:

<xao:getXML name="PostXAO" var="xml"/>

<xao:getXML name="PostXAO" var="xml">
	<?xml version="1.0"?>
	<parameters>
		<param name="type">getPostAndComments</param>
		<param name="id">1003768</param>
	</parameters>
</xao:getXML>



<xao:setXML name="PostXAO" value="${xmlValue}" />

<xao:setXML name="PostXAO">
	<?xml version="1.0"?>
	<test>
		<post>New Post</post>
	</test>
</xao:setXML>

public String getXML(String xml){
	
	boolean exp = xaoCache.expired(xml);
	
	if(exp){
	 	String returnXML = slowHttpGrabber.getAllMyShit();
	 	return returnXML;
	} else {
		return xaoCache.getXML(xml);
	}

}

	TransformXAO transformXAO = new TransformXAO();
	PostXAO postXAO = new PostXAO();
	
	String newXML = trasformXAO.getXML(postXAO.getXML());

Hiatus

Posted on Tuesday, July 29, 2003 04:10 PM

-Russ

Permalink | Comments [0]

Updating my Permalinks

Posted on Saturday, July 26, 2003 05:24 PM

Now today I'm going back to the most basic version - individual IDs per link like this: 1003702.html. Can you believe it? There are a few fundamental reasons. First, I want to include the comments on the permalinked page like MT does. I think it works well. Secondly, I want my Google results to be more relevant. Right now I talk about all sorts of topics per page, but Google smashes them all together so results like "FusionOne Resume" and "Kim Possible GBA ROM" end up on my website. Dividing the pages will narrow this down a bit more. And finally - on a Group weblog or one that includes aggregated content, there's a chance (that increases with the number of users) of two posts being created at the exact same moment, so using datetime just doesn't scale any more. IDs are ugly as hell, but they work. At least mine don't look like this: 1,6566,010,00.html (taken from the Nokia website - sufficiently unique for more pages than there are atoms in the universe).

I'm just hacking in the changes for now. Like everything else, I've got a 2/3 written rev of this weblogging software ready to go... but I'm sick of waiting for it, so I'm just slapping in the changes (and commenting out *tons* of SimpleDateFormatting code...).

Just goes to show - keeping it simple is usually the best route.

-Russ

P.S. My old permalinks should continue to work.

P.P.S. I'm also getting rid of the javascript for my comments, which more people have bitched about than you would believe.

Permalink | Comments [5]

Blogging GenCon

Posted on Saturday, July 26, 2003 03:01 PM

I was really into RPGs and especially RPG systems. In my storage at my parents I still have my collection of different gaming systems. I was totally into it. I even GMed at a conference in New Hampshire one year - it was pretty fun. It was an RPG called Toon which you can play cartoon characters. It sounds nuts, but in fact it's actually perfect for a conference setting because it's just a free for all with dice. 5 minutes to create a character, some sort of random plot and you let the players just do what they want. Anything goes as long as you can roll it.

Anyways, I'm not sure I'd be able to "let go" and enjoy a campaign any more... I'm too into real life. But still it's neat to read someone at GenCon - just like it was some technical conference, but each session is a new campaign! Rock! I'll have to Google to see if anyone else is blogging it as well...

-Russ

Permalink | Comments [0]

Python Needs a Corporate Angel

Posted on Friday, July 25, 2003 12:51 PM

This is not an indictment of Python as a whole. I discovered that it's as cool as I thought it was. It's crufty in some ways, but the important thing is that the Python community knows *exactly* where it's crufty, has documented it all and gone on with their lives. They get a little pissy being constantly questioned about those parts by newbies like myself, but in general the community is pretty cool and the power of Python easily makes up for any drawbacks.

However, here's the problem. Using Python when compared to Java is like using Linux when compared to Windows. Yes there's incredible benefits from the open source alternative, and great communities to support them, but there's an incredible variation when it comes to standards, documentation and quality of code. I know I ranted about the pressure of doing things the "right way" when it comes to Java recently, but I just didn't realize how much of a wild west development in Python is. When you're developing with Java you have the advantage of a corporate backed platform with a zillion books, articles, and a ton of official documentation about how *everything* works. There are few questions. Java is verbose and the corporate influence means that a bunch of corporate geeks love to find the most elaborate way to do anything in code before then declaring it the gospel (EJBs anyone?), but still... If I need to go to an IRC channel or search through mailing lists or newsgroups to find solutions to simple problems, I get annoyed. You might remember that Cocoon was the same exact way for me.

The thing that most disappointed me about Python is the lack of a standard web application server implementation. I looked at several different options and decided against going with all of them. The first was Webware. It's a really great clone of how a J2EE server works, but in Python. It has Python servlets, PSP pages, etc. It's very close to being like a Tomcat for Python. You can even plug in another project called Cheetah which works just like Velocity (I'm not sure which came first) for templating. You need to use Apache in front of Webware because it's not its own web server, but that's fine - it comes with an incredibly easy to install module for Apache which works quite well. If I got it set up, then you know it's pretty easy. However, once I got the basics working I was at a loss to figure out how to implement MVC! There doesn't seem to be any way for a servlet or PSP page to forward a request to another servlet or PSP page!! I'm not talking about a redirect, I'm talking about a server-side forward.

Ready? Here's the thing - there *might* be a way of doing it, but the documentation is really sketchy. That to me was a deal breaker.

The other ways I tried were mod_python and JonPy tools (which help with simple stuff like session management and db pooling) but just about *everyone* said "DON'T USE MOD_PYTHON!", I'm not even sure why, but I took them at their word. Besides, doing develpment that way would have been very low level and I wanted a bit more support than that. The third way I tried was Twisted - which I was really into for about a half hour. Once I grokked that I needed to do all the settings stuff in a Python file then call the twistd.py demon and pass that settings file to it to run, I quickly set up an http server with virtual sites that ran .rpy python scripts instantly! Cool! Except then I was ready to get into doing a DB call and I learned that Twisted is *single threaded*. What?!? You've GOT to be kidding me. If you want to do any sort of outside call, it's implemented as a callback. The code for a simple DB call was huge and not at all Pythonic. I'm sure Twisted is a wonderful project with a lot of advantages, but that was the deal breaker right there. Finally, I also checked out Draco and Aquarium and decided against those as well. (Call me picky).

So in my humble opinion, it would be great if a corporate entity - bigger and more influential than Zope - started pushing Python a bit more. Documentation, standards, application server reference implementations, etc. In other words, a bit more like Java. There's got to be a middle ground between tight corporate control and open source free-for-all. Corporate sponsership (say by IBM or even Intel or some company of that stature) would also give a boost to its use throughout the rest of the corporate world and maybe give a boost to the numbers of jobs looking for Python skills. The Python people who are happy with the status quo will probably think I'm nuts, and that's fine. I'm writing this for the teeming hoards of Java programmers looking around for something a bit funner (and more productive) to use who are going to run smack into the same issues I've run into.

Here's some advice for the Java guys moving over: The language is cool, but there's a lot of other supporting issues that you have to deal with. Where you can "live" in Java for the most part, Python will regularly drop down to the underlying platform. (Remember why you moved to Java in the first place?) It's mostly cross platform, but there's various libraries in the standard install that work only on Unix or only on Windows. When installing on Windows, many libraries come as .exes. There's no concept of .jars in Python, so when installing extensions, the actual .py files are copied into the C:\python\Lib\site-packages directory. There's also seems to be a dearth of specs - even database access doesn't seem to be particularly standard (like JDBC). And once you get outside the core libraries on Python.org, you can expect that anything goes in terms of code quality and documentation.

To the rabid Pythonistas who will assuredly jump down my throat on this... please educate me if I'm wrong and I'll post corrections, promise. These are just my thoughts after a couple weeks of learning Python.

One example where I was wrong before was that I slammed Jython as a half-ass Python implementation. Which is what it sort of looks like being 2 versions behind the current CPython specs. However, it's *not at all* half-ass, it's actually quite brilliant. This is just speculation, but I think that in the late 90s when Jython was being developed, Java was at the height of the hype curve and I think at one point there may have been a thought that Jython could be the successor to Python. A *lot* of work went into making Jython a really good Python implementation *and* the Python community even brought Jython into the fold, promoting it to basically even level with the original project. In books and articles now, you read about CPython and Jython as if they were equals, its quite interesting.

Jython needs help, however. Guido even petitioned help at the last Python event. It seems that there's a lot of users and not a lot of developers and now Jython is a few years behind where Python is on the development curve. Another problem with Jython besides it's libraries is that the code runs up to 10 times slower than its CPython and Java counter parts, yet I'm sure that there are ton of real heads out there that could improve that with optimization of the libraries... Personally I was disappointed that I couldn't just grab Mark's FeedParser.py file and run it in Jython. It'd be so nice to use that bit of code on my current project. Anyways, Jython is actually insanely cool, and I hope someone gets heads down on it soon so that it doesn't get farther behind.

That's all about this. I'm back in my comfy, yet frustratingly verbose Java world for now.

-Russ

Permalink | Comments [17]

Paranoia and Fear

Posted on Thursday, July 24, 2003 12:39 PM

And I wonder about the government's reach into online organizations. How many times has Amazon been petitioned for their book records? Has Google been tapped? Are they required to make a list of anyone searching for certain keywords and send it off to the FBI or Homeland security? How far has the government reached in to other online organizations?

I'm afraid of returning to the U.S. in so many ways. Remember - my wife is not a citizen and thus by the PATRIOT act she is a permanent suspect. The government can come and take her away for whatever reason they want and detain her for an unlimited amount of time. In fact it doesn't matter if you're a foreigner or not, anyone who criticizes the government is suspect as well ("If you're not with us you're against us"). Just the fact that this is true makes me afraid. Fear is the weapon of a dictatorial regime. I almost didn't write this.

George Orwell was right, just 20 years off on his dates.

-Russ

Permalink | Comments [15]

Sun Vs. Palm Continued

Posted on Tuesday, July 22, 2003 11:50 PM

ZDNet: The real opportunity for PalmSource is to tap in to the Java developer base because of its size-- roughly 3 million developers. Most if not all of them are experienced in building network-based applications, which is important as more and more handheld applications --- especially enterprise ones --- are expected to work over wireless networks with a lot of latency. Why doesn't PalmSource focus on one of the largest and fastest growing developer bases as opposed to the relatively small number of developers focused on the PalmOS? Also, many of ZDNet's readers have complained about how there are different versions of the PalmOS currently shipping with different PalmOS-based devices and how there are software incompatibilities between them. Why does that situation exist? The reason these questions are related is because, with Java as a development target, you get to move away from the question of whether it's Palm OS 4, Palm OS 5, or the next one.

Nagle: So, which version of Java do you have? Which implementation? The problem with Java is that it isn't unified. It really isn't. All the phone guys have their own variants. On my phone, I was hoping to use Java for the network interface; all the provisioning and transaction-based stuff. I don't want to rebuild that every time. I don't want to have to build a central socket for every other player. It would help to have some standardization, but all the carriers have forked off different variants of Java. So, I'm trying to be agnostic on this stuff.

ZDNet: That could be said of older versions of Java, but Sun and the Java Community Process seem committed to leveling that out. Many of those problems have either already gone away, or will be done away with eventually.

Nagle: They also seem committed to have differentiation. Everybody wants their variant to be the standard. Also, I don't know about the complaints and the confusion over different versions of the operating system. The applications that run on OS 4 work almost without exception on OS 5. It's been a pretty good transition and we have actually not had a lot of complaints.

...

ZDNet: I still think Java is a better direction. Based on what you've said so far, you're clearly trying to target and demonstrate some superiority over Microsoft and PocketPC. There's no one single development community that's more focused on that mission than the Java one. With PalmSource remaining focused on PalmOS, the phone guys working with Symbian and Java, and with RIM going Java all the way, Microsoft is succeeding at dividing and conquering. But, if you joined the battle, the outcome might be different. The other benefit is that Palm users immediately get access to a lot of Java applications that are portable from one device to the next. That's a strong selling point of Java--the compatibility of the applications with a variety of devices.

Nagle: Since the beginning of last year, we've gone from 12,500 applications available for the PalmOS platform to about 19,000 now. In terms of developers, we're up to 280,000. We're getting our share of I.Q. points focused on the platform, so we're not worried about lack of software. We have a large enough share of the market.

ZDNet: OK, why not have your cake and eat it too? You can stay with the PalmOS apps but still make it a great Java platform and then you could see which way the developers gravitate.

Nagle: I have no problem with that. I'd love to work with Sun. But Sun is very difficult to work with. They do not make it easy. We were one of the early members of the JCP. We tried to build a PDA profile, sort of a J2ME grown up a little bit. But we decided ourselves that that was sort of a bifurcation and that it created more problems than it solved. Sun didn't like it. They wouldn't support it, so we just decided that we weren't going to go through with it and that we would leave J2ME be.

ZDNet: Are you saying that PalmSource followed the rules of the JCP's framework and Sun vetoed it? Sun says it has never vetoed anything at the JCP.

Nagle: No, they didn't veto it. We actually got it approved. But in the end, there wasn't a lot of enthusiasm for promoting an extended platform. But I just want to address the attitude that you described where everything else goes away, operating systems become kernels, and everything runs on a runtime like Java. I just don't believe it.

The Zire 71 and Tungsten C gave me some technolust recently, but Palm is still doomed... this guy just doesn't get it. Over 1 million Symbian devices were sold in the second quarter in Europe alone (vs. 250,000 Palm OS devices).

Okay, back to my hole.

-Russ

Permalink | Comments [9]

I *Really* Want This Job

Posted on Tuesday, July 22, 2003 02:51 PM

US-CA-San Diego-Series 60 CDMA Solution Manager (technical marketing)

Position Description:
This position’s key tasks are to:
1. Create high level product descriptions and marketing material
· Produce marketing material to enable MSW Sales & Marketing effectively close deals
· Support MSW Marketing to define relevant marketing messages
· Act as a general interface and contact point between different parts of Series 60 CDMA Team and MSW Marketing & Sales
2. Ensure credible end-to-end (terminal and server software) and Series 60 CDMA eco system solution story
· Discuss new concepts with key CDMA carriers in cooperation with MSW CDMA Business Program and CDMA BU
· Implement product positioning based on business opportunities, threats and MSW strategies
3. Provide analysed customer feedback to improve product competitiveness
· Report customer and market status back to the Series 60 CDMA Team
· Clarify open issues
· Initiate development of concepts further
4. Help to define the solution marketing process and proposition
· Support MSW Sales to deliver the Series 60 CDMA message, address queries and close deals
5. Maintain customer awareness of Series 60 CDMA benefits
· Market the Series 60 CDMA solution based on customer information and solution needs
· Analyse competitor and market information
· Prepare and present material to maintain customer awareness
6. Enhance customer confidence in Nokia by managing customer commitments
· Visit key CDMA terminal vendors and CDMA carriers on a regular basis to ensure that customer satisfaction and good personal relatrionship is being maintained
· Work closely with the Series 60 CDMA R&D; people to ensure that customer expectations and requirements are being correctly addressed in the product line project plans
7. Assist the Series 60 CDMA Business Development in commercial matters such as:
· Time to market analysis,
· Manpower analysis
· Sales estimations
· Competitor Intelligence
· Interaction with other areas of the Series 60 CDMA organization

This role does not specifically have sub-ordinates. However, the person is expected to coach and guide less experienced product engineers from the business management team in their technical work.

Requirements:
A Bachelor or Master Degree in engineering, marketing or related field of study.
A thorough understanding of CDMA business activities, including:
· Awareness of CDMA terminal and operator business environment
Experience with:
· CDMA technology (important)
· CDMA Carriers (important)
· Series 60 (additional benefit)
· Symbian OS (additional benefit)
· Understanding of the mobile software project & product planning process

Anyone got any contacts, leads or any other inside knowledge that could help me get on the short list for this, I would be truly grateful.

Thanks,

-Russ

Permalink | Comments [4]

Out To Lunch

Posted on Monday, July 21, 2003 06:11 PM

-Russ

Permalink | Comments [4]

Neat Trillian Pro Trick

Posted on Tuesday, July 15, 2003 11:24 PM

It turns out that Trillian Pro includes a Word Matching option where you can add words to watch for and have events tied to it. Thus I'm now looking for "Russ". If anyone says my name, I get alerted. Poof. I can be part of the conversation without having to always be monitoring the channel. I *was* telling people to send me a private message so I could be alerted if I hid the window and something interesting was going on. Now I have a little customized message that pops up and says "someone's talking about you" if it sees my name in the chat. Neato!

Now I can just minimize the window and do other stuff, but still be easily contacted. I may add some other keywords as well, just for fun - it'd be neat to monitor several channels for certain keywords so you're always on top of the gossip. I'll have to play.

:-)

-Russ

Erik just told me that this is sort of a standard thing in IRC clients. I had *no* idea. It's still cool.

Permalink | Comments [0]

A Thought About Java and Python

Posted on Tuesday, July 15, 2003 10:49 PM

First let me explain the primal urge. Python attracts me because it doesn't seem like a "formal" programming language. Java is now used everywhere from universities to NASA and there are a zillion rights and wrongs that have been handed down from those on high and programming in Java now just makes me feel like I'm always doing something wrong. Hacking is downright encouraged in Python which is a fucking relief. Honestly, at this point in my education, I still prefer Java's overall logic that Python lacks, but I'm really attracted to Python's cleanness, relative simplicity and *complete lack of guilt* while using it.

But the problem is that Python is definitely crufty. There are so many spots where you can see how the language has been expanded - by force if necessary - throughout the years. Things like the way running an app works: you write a function that tests to see if matches "__main__", instead of an actual main() function. The fact that *every* method in a class needs to refer to "self" first (i.e.: method(self, otherParam) ) is ugly. And self isn't actually a reserved word, just a "strong convention". Ugh. As I learn more, I find more. In my opinion, Python could definitely use a new version rev, drop backwards compatability and clean up the cruft.

So you can see that I'm digging Python because of it's hackiness, but also disliking it for the same exact reason. And of course, I'm constantly comparing it to Java. Different languages do different things better, but as I run into areas where Java is so much better (db access, portability, packaging in jars) I wish I was back in the Java fold.

Basically, I can't help thinking that Java could be made to work more like Python. I mean, there's Jython, but I don't want to talk about half-ass implementing Python on Java, I'm talking about making Java more programmer friendly like Python.

This is already happening - but mostly on the server side of things. What I'm talking about is the various versions of JSP. The biggest flaw that most server side programmers have is to consider JSP a separate technology, when in reality, JSP is simply an elaborate macro language for creating servlet classes. Jasper parses JSP and creates a class. And that's the key - it doesn't matter *what* you program in using Java as long as the end result is a valid class file. This is the way Jython works: when the Python code is parsed and compiled, the result isn't a .pyc, but a regular .class file. Other examples include Apache's Axis project and their .jws files that produce all the classes that are needed for SOAP and WDSL and JSP 2.0's upcoming .tag files as well that produce tag libraries. All examples of non-full Java producing real classes.

So for all us non-programmers who could give a flying fuck about the GoF and just want to get something cool done in the shortest time and minimum lines possible, there needs to be another option than the increasingly complex and bloated Java, but not just on the server - for use on a daily basis.

I guess if it's not Java, it's not Java. However, something could be "mostly" Java and work more like Python. Would it be optimized? No. Would it be for the hard core geeks? No. Would be useful? Yeah, I think so. But I don't want some random OSS project that no one takes seriously, I'd like a sactioned Sun-backed spec like JSP.

Here's how I envision it. Imagine you had a new type of file called a .jv file. No compiling step needed. When you went to run it, the compiler would automatically be invoked, just like Python creating a normal Java class. There would be a set of libraries automatically imported that wrapped up the commonly used libraries in Java making things like i/o and Collections as simple to use as Python's Dictionaries, lists and open() function. Types would be dynamic, yet strongly typed and autoboxed (of course). It'd be easy to import existing libraries so you can take advantage of the massive amount of stuff produced for Java already, but not have to deal with the crap. Comments would have their own place, as well as neat functionality like overriding operators. No - I'm not asking for a VB for Java, just a version that's meant for higher productivity.

What do you think? I think if I never heard the term "anti-pattern" again, I'd be a happy man.

-Russ

Permalink | Comments [22]

Lunching With Moof

Posted on Tuesday, July 15, 2003 01:35 PM

Later... Lunch is over. It was good. I ate too much. Ugh. -Russ

Permalink | Comments [2]

Necho Stuff

Posted on Saturday, July 12, 2003 03:44 PM

Here's the latest Necho example. It's not bad, but there's some debate right now over the use or over-use of attributes and I left a comment with my thoughts on this. (This is what got me back into the wiki - there was a massive flux on this page in the past few days). Attributes are incredibly abused and misused (look at OPML as a prime example), however there's *got* to be a way of noting the variations of the same data types without using so many elements. For example, Title/subtitle, weblog/homepage/email/etc., created/issued/modified, and more. To me, <date type="issued"> is much clearer than just <issued>. I'm not sure what the answer is, but I think that Necho as it is now is a bit too verbose.

The best thing I found browsing the site this morning is this weblog called Formerly Echo, which "follows the development of the syndication framework formerly known as Echo". Very very neat! That's much better than the RSS feed which is just noise. Very cool- subscribed now.

Good stuff.

-Russ

Permalink | Comments [4]

The Gospel According to Jock

Posted on Friday, July 11, 2003 01:22 PM

... Procrastination is my biggest flaw, either I get obsessed by not wanting to screw up, or scared by not knowing how to begin. Either way I get paralyzed. I get easily sidetracked and distracted. I convince myself I am doing research, or building my skills. Then the longer I wait, the harder it gets to start.

The secret is to just get started. Doesn't matter where, it doesn't have to be right, just get started. Build up a head of steam. Build the smallest thing that is useable, then get people using the software. Then you have people demanding things be fixed and wanting features, it really pushes you forward. No matter what, just make sure you move forward every day. Joel on Software describes this in his Fire and Motion essay.

I haven't completely bought into the whole XP/Agile development thing, but some of it make sense. Streamlined process, and short development cycles keep you moving forward and more responsive to your customer. I suspect it's greatest value is that it makes it harder to procrastinate...

Stay tuned... something will happen soon, I'm sure.

-Russ

Permalink | Comments [9]

Personal Sys Admins

Posted on Thursday, July 10, 2003 10:33 PM

1. Download the RPM and do the usual rpm -ivh awstats.rpm (This will put files in /usr/local/awstats and /etc/awstats - you need to have Perl installed already.
2. Copy or make a symlink from /usr/local/awstats/wwwroot to your web directory
3. Make a copy of the /etc/awstats/awstats.virtualhost.conf and rename it for each virtual server you have, for example "awstats.www.russellbeattie.com.conf".
4. Edit the conf files to point at your logs. The conf is pretty straight forward.
5. From the command line, run the awstats.pl file that's in the wwwroot/cgi-bin directory with the -update flag. This should catch you up, though if there's a problem (like there was with mine) you may have to go into the conf and manually point at old log files to create the cached analyzed text.
6. Done! Check out the results from the web by something like /cgi-bin/awstats.pl

While Greg and I were talking about this, he was saying that he's definitely going to try it out, but is in the middle of messing with his email. He's doing filtering, spam detection and more and wanted to get that all worked out before messing with something else. And that's when I said, "I hate all this stuff. I want my own personal SysAdmin."

Hey yeah! Why not? Look, we have doctors, dentists and mechanics, no? They work on something that's ours and precious for a few hours when needed (our bodies, teeth and cars, respectively), why not our web servers as well? I'm not talking about some 16yo hacker in Albania (or maybe I am) but someone that somehow has credentials. Why hasn't someone hung out their shingle for this sort of thing already? Look at all these bloggers out there with their own servers just to run MT and have decent email, but with little time and less real experience to make their servers work like they should.

Well, I guess some of us like to get in there and hack our way around ourselves - like your Dad with that old Chevy in the back yard. But there's bound to be others out there who want an certified expert on hand, to hand them the keys (root access) and for them to fix it and bills us.

Seems like a future to me. In the 2000s, *everyone's* going to have their own personal server, no? Or is that just me? :-)

-Russ

Permalink | Comments [6]

Posts for August

• August, 2003
• July, 2003
• June, 2003
• May, 2003
• April, 2003
• March, 2003
• February, 2003
• January, 2003
• December, 2002
• November, 2002
• October, 2002
• September, 2002
• August, 2002
• July, 2002
• June, 2002
• May, 2002
• March, 2002
• February, 2002
• January, 2002
• December, 2001
• November, 2001
• October, 2001
• September, 2001