Domino talking Java 07 Jan, 2004
This one’s for you John :-). I’d like this ’blog entry to be a talking point, and hope others will chip in…
The other day, m’learned colleague John Barrow pinged me on Sametime (of course — see the Redbook wot he helped write!). We ended up discussing Java in Domino, specifically, DIIOP, i.e. remote invocation of the Domino Java classes and interfaces. Our collective experience of using DIIOP “in anger” is pretty minimal, but it looks as though it’s going to become more and more important as IBM move forward with Workplace, Domino tag libraries, portlets, and the like. So, the issue for the floor is not so much how well you’re getting on with Java / Java in Domino per se but how you’re actually using it. Local applets? Simple Java agents? A spot of networking? Separate apps talking to Domino? How stable do you find the DIIOP task in Domino? Is the DIIOP task happier in the ND6 environment?
(Addendum: in R5 this is assuming one has happily dealt with session leakages if that’s a concern — see Bob Balaban’s Domino Pool Manager product for more on this).
Also, I wonder whether the HTTP task is required for DIIOP or not… I guess I should check that.
So there it is: enquiring minds wish to know. I think this could be fruitful area of discussion as more and more enterprises look to integrate their Domino content and processes with the wider world of J2EE and / or .NET.
Here's some of the pains I've been having with DIIOP:
- Domino server behind firewall with NAT - solved using DIIOPIORHost=alternate_hostname_or_IP_address - it's a problem if the server has to serve DIIOP requests from both sides of the firewall
- Load-balanced Domino servers (using standard Network load balancing gear) - always use "sticky" connections for DIIOP, as Domino will not cope with load-balancing it
- DIIOP on Linux and LAZY programmers: default DIIOP port on Linux is 60148, not 63148
If I remember some more, I'll post them here.
NunoNuno Pereira#
-1-
A customer had a very old web-based application with an agent that generated the HTML page with quite some content from a Notes view (go guess). Anyway, they decided to expand the application to the whole company, so it moved from 15 users to about 500 concurrent users. Needless to say, pressing F5 was 3 minutes of wait guaranteed.
Instead of dwelling into the code and making the application more efficient (customer didn't want to invest too much time/money in it) we simply rewrote that agent as a Java Servlet.
The reload time per page is now about 15 seconds, mainly due (I guess) to not having the Opening-loading-running-closing overhead that a LotusScript agent has.
-2-
A few years ago, while following some internal LPS training on some new-fantastic-and-never-released Web Profiling solution for Domino that ran entirely in Java in Domino (Servlets and agents) the instructor did a simple test for speed:
He wrote 2 agents with basically the same code:
" For x = 1 to 1000000 step 1 "
One of the agents took about 20 seconds to complete, the other took nearly 25 minutes. Do I really need to say which agent was written in LotusScript? ;-)
NunoNuno Pereira#
HTTP is not required to access DIIOP on ND6 !!!
I'm intrigued by the implication suggested above that workplace will integrate with Domino via DIIOP. If this is actually the case, it impacts an ongoing discussion in my organization whether to migrate data to DB2 to provide more robust reporting facilitites.
A lotus.domino.viewentrycollection can be just as effective as an SQL query, provided you access column values, versus opening each document individually.
Does anyone know whether Workplace is using DIIOP to access ND ??Neill Laney#
As for Workplace and Domino integrating via DIIOP, that’s supposition on my part, I haven’t looked into the architectural requirements of this too much. All I can think of is the current Domino Toolkit for WSAD, with the Domino tag library: how do JSP tags reference Domino objects otherwise? Surely it can’t rely on the Domino and J2EE servers being on the same box?
Any further insight would be much appreciated I’m sure — meanwhile, I should dig some more detail up.Ben Poole#
i've been thinking a bit more about that "bridge class" for eclipse debugging. still just kind of messing around with eclipse, mostly using it right now just to work in a real ide, and to become familiar with it, but i haven't tried to debug any code yet.
anyway, i was thinking, how would i get a handle to a real session on a server in an r5 environment, and get agentcontext and all that. anyone got an idea where to start? i'm just wondering if anyone has already done it, in eclipse or whatever.
-->end bunny trail
so far we are doing java agents and java servlets (to domino-only data). we are looking increasingly at web services and xml for little things here and there and thinking about how we can do all that with java. a little tricky with r5 with the old jvm. i print out quite a bit of html in production databases with java and it runs very fast.
the code is optimized (minimizing getNths and using column values and creating as much html in the document ahead of time as i can and passing that into the view columns) and so forth. been very happy with the flexibility and the speed, and looking to move more code to servlets soon.
since we hadn't done much with servlets we were a little nervous about it, esp after some comments by bob balaban about the stability of the r5 servlet container, but i'm getting more comfortable. need to build a little tool to push code around so that we can "replicate" the code up when we have changes. once i get that done i'll probably move as much as i can to servlets. jonvon#
I think that generating a “bridge” in Eclipse is going to be a problem. Try this on for size, and then please feel free to pick holes in my argument ;-) :
AgentContextis determined in a way particular to the Notes client and its JVM… (like I say, I’m on iffy ground here, so if someone wants to correct me, please do!), i.e. we can’t get a handle on the agent context when we move away from the Notes Java environment unless it’s passed through from Notes.No agent context means no way of debugging said code from the outside. So, to get a handle on a session from external code means using the
NotesFactory. I wonder if I’ve got this right…Ben Poole#the classes i've been working on lately all take a session object as a required argument in the constructor. maybe that is the key, if you always pass the session through that way, then anything you are working on outside of designer should be able to run just fine, basically (jvm's and whatnot aside). whether the session comes from notesfactory or from agentcontext just doesn't matter if the code is constructed the right way.
so the idea would be, write code that is independent of the container, whether it's a servlet or an agent or something else, do debugging via something that grabs a session via notesfactory, then use your code appropriately in whatever context you toss it into.
thanks for letting me think out loud here. :-)jonvon#
I ran a stress test against my serlvet (open NAB, display fullnames of all documents in People view), running under Tomcat on the same server as Domino, with 50 conucrrent users and Domino had a "Panic" within about 30 seconds of a 1 minute load test. This isn't just a friendly error or something, it was Domino physically crashing ungracefully.
While inefficient, the native access method (sinitThread/stermThread on every doGet) coped relatively well considering the server was a desktop box. Of course in the "real world" your Websphere server is unlikely to be the same box as your Domino server so DIIOP is your only option.
Caching connections and stuff is fine, and might prevent crashing under load, but what about when you can't use anonymous / generic user DIIOP and need to authenticate with individual Domino users? While it didn't crash, it was pathetically slow with 50 invidiual DIIOP connections.
Maybe I was just doing something wrong, but I found a couple of Domino servlet examples and they were pretty much coded the same way mine was. I was intending to write this all up and post it somewhere, but I didn't get around to it and now there's actually work to do again… :)Marcin#
And also thought I should mention that all this was on Domino 6.03.Marcin#
Thus far, the only way I've used Java in a Notes/Domino context that didn't send me screaming was as standalone apps. Access to a decent IDE, debugger, non-paleolithic JVMs, and stable servers being why.Colin Pretorius#
With Next Gen having gone the way of WP and tag libraries, it stands to reason the task must be considered up to the job, and in the queue for additional improvements.
Certainly, I've run into method calls that will literally crash the server, but these are isolated bugs, that can be pinpointed using a decent IDE and debugger like WSAD, or even the Sysdeo plugin for Eclipse, if you can get it to work. (That's enough to send me screaming). AFAIK, the trial versions of WSAD available from the Developer Domain are full versions, for developer use.
My overtime task for the day is to load up a view with documents and access the column views for display in a JSP (I'm not a big fan of tags). It's doubtful I can do this in time to contribute further to this discussion, but if you aren't going to use DIIOP, and local calls are impractical, the alternative is using nhhtp (guaranteed to send one screaming), replicating data to a relational db, or migrating. My desire is to use DIIOP because it is an elegant solution, if nothing else.Neill Laney#
That said, everyone who’s mentioned dodgy DIIOP calls has also mentioned that this is on R5, which is fair enough (and to be expected given that it’s arguably the version most of us have more experience with). My original post isn’t aimed at one version or t’other, as we currently only use R5 in production: a better implementation of DIIOP would certainly aid the argument for moving to 6.Ben Poole#
As it stands I think everyone’s doing OK. I wonder which posts Neill takes issue with when he says this:
Ben Poole#
Basically, returning up 1000 records in a view with 10 columns yielded satisfactory results with 50 concurrent users run 3 times with 5 secs between runs. Granted, I'm not using a very sophisticated methodology, but I'm looking for a reporting solution, and this will work fine for the limited number of power users I'm supporting.
Here's the results
One observation I might point out about Marcin's test is that Tomcat likely contributed to the low load performance.Neill Laney#
A colleague of mine pointed out the phenominal improvement in performance when returning lotus.domino.ViewEntry.columnValues instead of accessing the document to obtain the values.
Thanks for thread. Had it not been for the challenge of backing up my opinion, I wouldn't have performed the test, out of sheer laziness. One more thing I want to check, and that is to seperate the logic of iterating through a collection to output every value in a JSP from instantiating and building the collection, then returning some operation on the collection for ouput.
The very basic web app I'm using for the test, using a JSP for presentation, with a usebean tag which creates a session over DIIOP, a ViewEntryCollection, and returns an ArrayList of Vectors corresponding to the columnValues in the View. I think it would be useful to isolate the access time to return the ArrayList, and perform a count of the columns or some other nominal operation for comparison to the results I posted previously. In fact, it's an oversight on my part not to have done this in the first place. However, I expect to acheive similar results.
The Notes db has one form with 10 text fields and one view with 10 columns. For this test, the db contained 5000 documents in the view with field values from "MyField_1" to "MyField_49999".
My experience is that result sets using DIIOP aren't cached, so when changes to the documents underlying the ViewEntryCollection are saved, the result set is changed. I believe this is a valid methodology and interested in other's opinions here. Thanks, again, for the thread and the forum, Ben!Neill Laney#
I would be willing to take a look at your code and details on the panic and provide a meaningful answer.
I also would like to point out this article we wrote about DIIOP and the Java back-end classes which should help answer your questions.
http://www-10.lotus.com/ldd/today.nsf/Lookup/Java_access_pt1Steve Nikopoulos#
In this testing I found IIOP was stable. I suppose with a proper "production" quality server and separate box for the Resin server performance would improve - but I don't have the resources to try that :)
To be fair the test I was doing that caused the panic previously was creating a DIIOP session on every request (and recycling it at the end). The reason I was doing this was to simulate what would happen in an environment where every user needed to be authenticated. What is the correct "pattern" in this case, where the each request is potentially by a different Domino user and you potentially have several hundred simultaneous users?
Thanks seeding this thread Ben, it's been very interesting! :)Marcin#
Just kidding! John is great, but then I haven’t had any Liverpool FC-related examples in any of our meetings yet ;-)
Now then, today John delivered a copy of the red book you chaps did: splendid. But My God, it wasn’t signed!Ben Poole#
What finally did the trick was using the DIIOPIgnorePortLimits=1 notes.ini configuration. So far, so good (breath held).
Thanks again.
Randy
Randy K#
For example, Notes might actually do exactly what you tell it to do - spin through 1000000 iterations, regardless of what's inside the loop. But I believe that Java would smartly recognize that nothing's actually being performed in the loop, and skip the operation entirely. I'm not certain about that, but I believe I've read some interesting things, like this, about what kind of code optimization occurs with java. I hope your instructor pointed out this as a possibility.
I'd be much more impressed if similar operations dealing with similar domino objects was performed inside of the loop.
Yes, I do believe that Java is much faster at most things, but I'm not certain, without attempting my own tests, if accessing all kinds of Domino objects, using the same methods (e.g. LS -
NotesView.GetNextDocument(NotesDocument)compared to Java -lotus.domino.View.getNextDocument(lotus.domino.Document)would yield similar results.I'm really just getting started with using Java with Domino (frankly, because I'm scared to death that I'm going to forget a
.recycle()method somewhere, and run out of memory on the server), so I'm not certain I can really accurately perform the tests myself.Steve Chapman#Gyan#