PreviousNext…

Domino talking Java

This one’s for you John :-). I’d like this ’blog entry to be a talking point, and hope others will chip in…

The other day, m’learned colleague John Barrow pinged me on Sametime (of course — see the Redbook wot he helped write!). We ended up discussing Java in Domino, specifically, DIIOP, i.e. remote invocation of the Domino Java classes and interfaces. Our collective experience of using DIIOP “in anger” is pretty minimal, but it looks as though it’s going to become more and more important as IBM move forward with Workplace, Domino tag libraries, portlets, and the like. So, the issue for the floor is not so much how well you’re getting on with Java / Java in Domino per se but how you’re actually using it. Local applets? Simple Java agents? A spot of networking? Separate apps talking to Domino? How stable do you find the DIIOP task in Domino? Is the DIIOP task happier in the ND6 environment?

(Addendum: in R5 this is assuming one has happily dealt with session leakages if that’s a concern — see Bob Balaban’s Domino Pool Manager product for more on this).

Also, I wonder whether the HTTP task is required for DIIOP or not… I guess I should check that.

So there it is: enquiring minds wish to know. I think this could be fruitful area of discussion as more and more enterprises look to integrate their Domino content and processes with the wider world of J2EE and / or .NET.

Comments

  1. HTTP is required for DIIOP to work correctly. When you connect using DIIOP there's an initial HTTP GET done to some file (can't remember the correct name now) to find out the port you're using for DIIOP.

    Here's some of the pains I've been having with DIIOP:

    - Domino server behind firewall with NAT - solved using DIIOPIORHost=alternate_hostname_or_IP_address - it's a problem if the server has to serve DIIOP requests from both sides of the firewall
    - Load-balanced Domino servers (using standard Network load balancing gear) - always use "sticky" connections for DIIOP, as Domino will not cope with load-balancing it
    - DIIOP on Linux and LAZY programmers: default DIIOP port on Linux is 60148, not 63148

    If I remember some more, I'll post them here.

    NunoNuno Pereira#
  2. Thanks Nuno, that’s what I thought re DIIOP and HTTP. That port thing’s a gotcha too, unless coders actually soft-code everything… something that is a never-ending struggle eh? ;-)Ben Poole#
  3. Regarding the "how you’re actually using it" part, here's some good examples of where and when to use Java IN Notes.

    -1-

    A customer had a very old web-based application with an agent that generated the HTML page with quite some content from a Notes view (go guess). Anyway, they decided to expand the application to the whole company, so it moved from 15 users to about 500 concurrent users. Needless to say, pressing F5 was 3 minutes of wait guaranteed.

    Instead of dwelling into the code and making the application more efficient (customer didn't want to invest too much time/money in it) we simply rewrote that agent as a Java Servlet.
    The reload time per page is now about 15 seconds, mainly due (I guess) to not having the Opening-loading-running-closing overhead that a LotusScript agent has.

    -2-

    A few years ago, while following some internal LPS training on some new-fantastic-and-never-released Web Profiling solution for Domino that ran entirely in Java in Domino (Servlets and agents) the instructor did a simple test for speed:
    He wrote 2 agents with basically the same code:
    " For x = 1 to 1000000 step 1 "
    One of the agents took about 20 seconds to complete, the other took nearly 25 minutes. Do I really need to say which agent was written in LotusScript? ;-)

    NunoNuno Pereira#
  4. I've done a considerable amount of working with DIIOP in non-production environs, without any major problems with the task.

    HTTP is not required to access DIIOP on ND6 !!!

    I'm intrigued by the implication suggested above that workplace will integrate with Domino via DIIOP. If this is actually the case, it impacts an ongoing discussion in my organization whether to migrate data to DB2 to provide more robust reporting facilitites.

    A lotus.domino.viewentrycollection can be just as effective as an SQL query, provided you access column values, versus opening each document individually.

    Does anyone know whether Workplace is using DIIOP to access ND ??Neill Laney#
  5. Interesting re the HTTP requirement in ND6. I place emphasis on this previous requirement as a possible issue because a lot of organisations are still R5-based. However, it’s great that ND6 removes this dependency.

    As for Workplace and Domino integrating via DIIOP, that’s supposition on my part, I haven’t looked into the architectural requirements of this too much. All I can think of is the current Domino Toolkit for WSAD, with the Domino tag library: how do JSP tags reference Domino objects otherwise? Surely it can’t rely on the Domino and J2EE servers being on the same box?

    Any further insight would be much appreciated I’m sure — meanwhile, I should dig some more detail up.Ben Poole#
  6. -->begin bunny trail
    i've been thinking a bit more about that "bridge class" for eclipse debugging. still just kind of messing around with eclipse, mostly using it right now just to work in a real ide, and to become familiar with it, but i haven't tried to debug any code yet.

    anyway, i was thinking, how would i get a handle to a real session on a server in an r5 environment, and get agentcontext and all that. anyone got an idea where to start? i'm just wondering if anyone has already done it, in eclipse or whatever.
    -->end bunny trail

    so far we are doing java agents and java servlets (to domino-only data). we are looking increasingly at web services and xml for little things here and there and thinking about how we can do all that with java. a little tricky with r5 with the old jvm. i print out quite a bit of html in production databases with java and it runs very fast.

    the code is optimized (minimizing getNths and using column values and creating as much html in the document ahead of time as i can and passing that into the view columns) and so forth. been very happy with the flexibility and the speed, and looking to move more code to servlets soon.

    since we hadn't done much with servlets we were a little nervous about it, esp after some comments by bob balaban about the stability of the r5 servlet container, but i'm getting more comfortable. need to build a little tool to push code around so that we can "replicate" the code up when we have changes. once i get that done i'll probably move as much as i can to servlets. jonvon#
  7. Hi jonvon! Thanks for chipping in.

    I think that generating a “bridge” in Eclipse is going to be a problem. Try this on for size, and then please feel free to pick holes in my argument ;-) :

    AgentContext is determined in a way particular to the Notes client and its JVM… (like I say, I’m on iffy ground here, so if someone wants to correct me, please do!), i.e. we can’t get a handle on the agent context when we move away from the Notes Java environment unless it’s passed through from Notes.

    No agent context means no way of debugging said code from the outside. So, to get a handle on a session from external code means using the NotesFactory. I wonder if I’ve got this right…Ben Poole#
  8. that's what i've been thinking too, something along those lines. i've no idea how the mechanics of agentcontext really come together, as to whether it is actually dependent on the code running inside an agent.

    the classes i've been working on lately all take a session object as a required argument in the constructor. maybe that is the key, if you always pass the session through that way, then anything you are working on outside of designer should be able to run just fine, basically (jvm's and whatnot aside). whether the session comes from notesfactory or from agentcontext just doesn't matter if the code is constructed the right way.

    so the idea would be, write code that is independent of the container, whether it's a servlet or an agent or something else, do debugging via something that grabs a session via notesfactory, then use your code appropriately in whatever context you toss it into.

    thanks for letting me think out loud here. :-)jonvon#
  9. Over the slightly slower Christmas / New Year period I did some "R&D" on Java Servlets accessing Domino. Essentially, DIIOP is a flaky piece of crap under load.

    I ran a stress test against my serlvet (open NAB, display fullnames of all documents in People view), running under Tomcat on the same server as Domino, with 50 conucrrent users and Domino had a "Panic" within about 30 seconds of a 1 minute load test. This isn't just a friendly error or something, it was Domino physically crashing ungracefully.

    While inefficient, the native access method (sinitThread/stermThread on every doGet) coped relatively well considering the server was a desktop box. Of course in the "real world" your Websphere server is unlikely to be the same box as your Domino server so DIIOP is your only option.

    Caching connections and stuff is fine, and might prevent crashing under load, but what about when you can't use anonymous / generic user DIIOP and need to authenticate with individual Domino users? While it didn't crash, it was pathetically slow with 50 invidiual DIIOP connections.

    Maybe I was just doing something wrong, but I found a couple of Domino servlet examples and they were pretty much coded the same way mine was. I was intending to write this all up and post it somewhere, but I didn't get around to it and now there's actually work to do again… :)Marcin#
  10. And addendum to my last post, after discussing this with my workmates. I also tried the JSP Tag Libraries. Again, same sort of thing - a simple JSP page that connected to the NAB and go the Full Names of all the People in it. That page would take about 5 seconds to return anything every time you hit refresh - imagine if I was actually doing something complex!

    And also thought I should mention that all this was on Domino 6.03.Marcin#
  11. My experience with DIIOP has been rather negative. It's a fraction of the speed of 'local' access and crashed my R5 server on Linux in no time.

    Thus far, the only way I've used Java in a Notes/Domino context that didn't send me screaming was as standalone apps. Access to a decent IDE, debugger, non-paleolithic JVMs, and stable servers being why.Colin Pretorius#
  12. I'm disappointed no distinction is being made here between DIIOP on R5 and ND6. At least in the notes.net forum, you're posting in one or the other forum, and would get a nice rebuff for discussing one version or the other in the wrong forum.

    With Next Gen having gone the way of WP and tag libraries, it stands to reason the task must be considered up to the job, and in the queue for additional improvements.

    Certainly, I've run into method calls that will literally crash the server, but these are isolated bugs, that can be pinpointed using a decent IDE and debugger like WSAD, or even the Sysdeo plugin for Eclipse, if you can get it to work. (That's enough to send me screaming). AFAIK, the trial versions of WSAD available from the Developer Domain are full versions, for developer use.

    My overtime task for the day is to load up a view with documents and access the column views for display in a JSP (I'm not a big fan of tags). It's doubtful I can do this in time to contribute further to this discussion, but if you aren't going to use DIIOP, and local calls are impractical, the alternative is using nhhtp (guaranteed to send one screaming), replicating data to a relational db, or migrating. My desire is to use DIIOP because it is an elegant solution, if nothing else.Neill Laney#
  13. As far as the distinction goes, that’s what I want to get more of a handle on, personally. All I know at this stage is that ND6 is “better” and that R5 is “flaky”. More detail from those who have experience is appreciated, believe me!

    That said, everyone who’s mentioned dodgy DIIOP calls has also mentioned that this is on R5, which is fair enough (and to be expected given that it’s arguably the version most of us have more experience with). My original post isn’t aimed at one version or t’other, as we currently only use R5 in production: a better implementation of DIIOP would certainly aid the argument for moving to 6.Ben Poole#
  14. Ben - My comments regarding JSP tag library performance and DIIOP crashing under load were on Domino 6.03, as I mentioned in my follow up post.Marcin#
  15. My apologies Marcin, I was overlooking your post, and I didn’t intend that at all.

    As it stands I think everyone’s doing OK. I wonder which posts Neill takes issue with when he says this:
    I’m disappointed no distinction is being made here between DIIOP on R5 and ND6
    Ben Poole#
  16. I performed my own stress test using the WebSphere Test Environment (WTE) in WSAD 5.1 on a beefy W2K PC and a slightly less beefy W2k PC running Domino 6.02 CF2.

    Basically, returning up 1000 records in a view with 10 columns yielded satisfactory results with 50 concurrent users run 3 times with 5 secs between runs. Granted, I'm not using a very sophisticated methodology, but I'm looking for a reporting solution, and this will work fine for the limited number of power users I'm supporting.

    Here's the results

    One observation I might point out about Marcin's test is that Tomcat likely contributed to the low load performance.Neill Laney#
  17. Thanks Neill, that’s useful. I would regard Tomcat as more of a “reference platform” than something to be used as a production server, although I know some people do use it for sites etc. (e.g. Matt Raible).Ben Poole#
  18. I think it's a matter of using what's available, and WAS is overkill for the majority of sites out there with a maximum concurrent number of users of 1, like mine.

    A colleague of mine pointed out the phenominal improvement in performance when returning lotus.domino.ViewEntry.columnValues instead of accessing the document to obtain the values.

    Thanks for thread. Had it not been for the challenge of backing up my opinion, I wouldn't have performed the test, out of sheer laziness. One more thing I want to check, and that is to seperate the logic of iterating through a collection to output every value in a JSP from instantiating and building the collection, then returning some operation on the collection for ouput.

    The very basic web app I'm using for the test, using a JSP for presentation, with a usebean tag which creates a session over DIIOP, a ViewEntryCollection, and returns an ArrayList of Vectors corresponding to the columnValues in the View. I think it would be useful to isolate the access time to return the ArrayList, and perform a count of the columns or some other nominal operation for comparison to the results I posted previously. In fact, it's an oversight on my part not to have done this in the first place. However, I expect to acheive similar results.

    The Notes db has one form with 10 text fields and one view with 10 columns. For this test, the db contained 5000 documents in the view with field values from "MyField_1" to "MyField_49999".

    My experience is that result sets using DIIOP aren't cached, so when changes to the documents underlying the ViewEntryCollection are saved, the result set is changed. I believe this is a valid methodology and interested in other's opinions here. Thanks, again, for the thread and the forum, Ben!Neill Laney#
  19. Not at all Neill, thanks for your valuable contributions: you’ve inspired me to break out Tomcat again so I can start a-tinkering… ;-)Ben Poole#
  20. Regarding this comment: "Over the slightly slower Christmas / New Year period I did some "R&D" on Java Servlets accessing Domino. Essentially, DIIOP is a flaky piece of crap under load."

    I would be willing to take a look at your code and details on the panic and provide a meaningful answer.

    I also would like to point out this article we wrote about DIIOP and the Java back-end classes which should help answer your questions.
    http://www-10.lotus.com/ldd/today.nsf/Lookup/Java_access_pt1Steve Nikopoulos#
  21. After Neill mentioned that Tomcat might be the source of my performance issues I downloaded and installed Resin (http://www.caucho.com/) which is supposed to be a high performance JSP/Servlet container. I also rewrote the servlet to use ViewEntry stuff (the test servlet just displays all the names in the test server's address book, about 30 names). I re-ran my tests and found:
    • For local access (sinitThread) I could get about 35 requests per second (vs about 25/sec for reading from docs)
    • For IIOP access (same server, one anonymous session, creating DB and View handle in servlet init) I got 5 requests per second

    In this testing I found IIOP was stable. I suppose with a proper "production" quality server and separate box for the Resin server performance would improve - but I don't have the resources to try that :)

    To be fair the test I was doing that caused the panic previously was creating a DIIOP session on every request (and recycling it at the end). The reason I was doing this was to simulate what would happen in an environment where every user needed to be authenticated. What is the correct "pattern" in this case, where the each request is potentially by a different Domino user and you potentially have several hundred simultaneous users?

    Thanks seeding this thread Ben, it's been very interesting! :)Marcin#
  22. Friends with John Barrow, you really need to get out more :-) He's a good guy, and he did some great chapters in that redbook, even if they took Liverpool FC out of his biography [smiley frown] Carl #
  23. Hey, I only said he was a colleague…

    Just kidding! John is great, but then I haven’t had any Liverpool FC-related examples in any of our meetings yet ;-)

    Now then, today John delivered a copy of the red book you chaps did: splendid. But My God, it wasn’t signed!Ben Poole#
  24. These comments were invaluable to me as I tried to get DIIOP up and running on a ND6.0.3 Linux-HA cluster.

    What finally did the trick was using the DIIOPIgnorePortLimits=1 notes.ini configuration. So far, so good (breath held).

    Thanks again.
    Randy
    Randy K#
  25. Just a response to Nuno Pereira's message (#3.2), at first it would seem that Java's just MUCH faster at doing everything from what your instructor showed you, however, I believe that this particular example really only shows that the compiler (or rather Interpreter?) for java, does a better job at optimizing certain code.

    For example, Notes might actually do exactly what you tell it to do - spin through 1000000 iterations, regardless of what's inside the loop. But I believe that Java would smartly recognize that nothing's actually being performed in the loop, and skip the operation entirely. I'm not certain about that, but I believe I've read some interesting things, like this, about what kind of code optimization occurs with java. I hope your instructor pointed out this as a possibility.

    I'd be much more impressed if similar operations dealing with similar domino objects was performed inside of the loop.

    Yes, I do believe that Java is much faster at most things, but I'm not certain, without attempting my own tests, if accessing all kinds of Domino objects, using the same methods (e.g. LS - NotesView.GetNextDocument(NotesDocument) compared to Java - lotus.domino.View.getNextDocument(lotus.domino.Document) would yield similar results.

    I'm really just getting started with using Java with Domino (frankly, because I'm scared to death that I'm going to forget a .recycle() method somewhere, and run out of memory on the server), so I'm not certain I can really accurately perform the tests myself.Steve Chapman#
  26. from where i can download lotus.domino package containing java classes to communicate with the domino server???
    Gyan#
  27. They come with it.Ben Poole#

Comments on this post are now closed.

About

I’m a developer / general IT wrangler, specialising in (often mobile-friendly) web apps using things like node.js, Java, C#, PHP, HTML5, etc.

Best described as a simpleton, but kindly. Read more…