CPS 12.0.11 – Intermittent freezing while placing orders – Bugs, Known Issues, Open Cases

rchildress

(@rchildress)

Posts: 13

Active Member

Topic starter

We upgraded to CPS 12.0.11 last week and are experiencing intermittent client freezes while placing orders in a patient's chart. When a caregiver checks something in the order sheet, it will occasionally freeze the entire client. Nothing is reported to the standard client logs, on the application server, or on the database server. Windows error reporting never record anything, either. The only way to get the affected users back to work is to run the "Reset CPS Process" batch file or Task Manager.

This does not happen every time an order is made. Instead, the issue seems to crop up just to disappear several minutes later and does this ALL DAY. Has anyone else experienced this issue in SP11 or any previous versions of CPS 12.0?

UPDATE (Jan 14, 2016):
Please see this post for a temporary workaround to this issue until a proper fix is provided by GE Healthcare.

Posted : December 18, 2015 11:16 am

aballard

(@aballard)

Posts: 43

Trusted Member

We upgraded to SP11 last week and this started happening to us this week. Working with our VAR on this issue, but we have not found a cause yet.

I restarted jboss in the middle of it acting up but the issue continued as soon as the server came back online.

Posted : December 18, 2015 11:36 am

rchildress

(@rchildress)

Posts: 13

Active Member

Topic starter

We've tried restarting the servers, reviewed windows updates, made debug tracking logs, and have spent two days on the phone with support.

They were able to record the issue once, so they know it is credible. Just can't get any insight from anywhere as to what is crashing or freezing and why.

Posted : December 18, 2015 12:00 pm

aballard

(@aballard)

Posts: 43

Trusted Member

Richard, It sounds like your further along in troubleshooting than us. Do you have a GE case # that you would mind sharing? My VAR is going to call GE to see if others are experiencing this issue (obviously there are).

We have ran sql traces, mel traces, checked for expensive queries and excessive disk IO on SQL server.

JBoss server appears healthy. We haven't restarted SQL yet.

Happens across multiple terminal servers and workstations.

Posted : December 18, 2015 12:23 pm

rchildress

(@rchildress)

Posts: 13

Active Member

Topic starter

Certainly. I believe it is 1-795559275. Sounds like you've been able to get more logs than me.

What OSes are the SQL server and jBoss servers running on? Also, are your Windows Updates current? Virtualized or no?

I can confirm that this is happening on fat and thin clients with both wired and wireless networking.

We have Microsoft SQL Server 2012 and jBoss running on two VMware guests with Windows Server 2012 and the latest important Windows Updates.

Posted : December 18, 2015 12:30 pm

aballard

(@aballard)

Posts: 43

Trusted Member

I should add that I saw some invalid row errors in the db trace when adding the diagnosis to the order- the tech didn't see concerned about the errors.

12/17/2015 11:36:57.756 INFO Process Id #166652 ML Thread [Trace] 74166886> SQL Open Slave End row: 0
12/17/2015 11:36:57.756 INFO Process Id #166652 ML Thread [Trace] 74166886> LargeList Refill In-memory Cache Tbl: Provider MyFolder
12/17/2015 11:36:57.756 INFO Process Id #166652 ML Thread [Trace] 74166886> SQL Close Object: LLFindUserMFCacheTbl
12/17/2015 11:36:57.756 INFO Process Id #166652 ML Thread [Trace] 74166886> SetCurrentRow Begin old: -1 request: -1 fire: 1 object: LLFindUserMFCacheTbl
12/17/2015 11:36:57.756 INFO Process Id #166652 ML Thread [Trace] 74166886> SetCurrentRow End old: -1 request: -1 new: -1 fire: 1 object: LLFindUserMFCacheTbl
12/17/2015 11:36:57.756 INFO Process Id #166652 ML Thread [Trace] 74166886> CMLFindDlg::fillComboTbl, loadMFfromPrefs(), reload MyFolder cache.
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> SetCurrentRow Begin old: -1 request: 75 fire: 1 object: _MasterOrdDx
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Formula (Triggered) Begin: VPTEXT (Caused By) Argument: ORDDXID
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: DXDESC
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: DXDESC: Value: SPONDYLOSIS WITH RADICULOPATHY CERVICAL;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: TMPPRIORITY
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: TMPPRIORITY: Value: 1;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: TMPINDENT
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: TMPINDENT: Value: 0;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: DXCODE
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: DXCODE: Value: ICD-M47.22;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: ICD9MASTERDIAGNOSISID
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: ICD9MASTERDIAGNOSISID: Value: 732029;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: ICD10MASTERDIAGNOSISID
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: ICD10MASTERDIAGNOSISID: Value: 352277;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: SNOMEDMASTERDIAGNOSISID
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: SNOMEDMASTERDIAGNOSISID: Value: ;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Execute Begin: VPTEXT
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Execute End: VPTEXT: Value: SPONDYLOSIS WITH RADICULOPATHY CERVICAL;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Set Begin: VPTEXT: Value: SPONDYLOSIS WITH RADICULOPATHY CERVICAL;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Set End: VPTEXT: Value: SPONDYLOSIS WITH RADICULOPATHY CERVICAL;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Formula End: VPTEXT
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> SetCurrentRow End old: -1 request: 75 new: 75 fire: 1 object: _MasterOrdDx
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Clear Cache: VPTEXT
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> SetCurrentRow Begin old: 75 request: 76 fire: 1 object: _MasterOrdDx
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Formula (Triggered) Begin: VPTEXT (Caused By) Argument: ORDDXID
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: DXDESC
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> INVALID ROW
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg (Error) End: DXDESC;
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: TMPPRIORITY
12/17/2015 11:36:57.802 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: TMPPRIORITY: Value: 1;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: TMPINDENT
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: TMPINDENT: Value: 0;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: DXCODE
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> INVALID ROW
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg (Error) End: DXCODE;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: ICD9MASTERDIAGNOSISID
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> INVALID ROW
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg (Error) End: ICD9MASTERDIAGNOSISID;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: ICD10MASTERDIAGNOSISID
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> INVALID ROW
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg (Error) End: ICD10MASTERDIAGNOSISID;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: SNOMEDMASTERDIAGNOSISID
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> INVALID ROW
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg (Error) End: SNOMEDMASTERDIAGNOSISID;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Execute Begin: VPTEXT
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Execute End: VPTEXT: Value: ;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Set Begin: VPTEXT: Value: ;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Set End: VPTEXT: Value: ;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Formula End: VPTEXT
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> INVALID ROW
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> SetCurrentRow End old: 75 request: 76 new: -1 fire: 1 object: _MasterOrdDx
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Formula (Triggered) Begin: _ORDERDIAGNOSES (Caused By) Argument: DXGROUPID
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg Begin: DXGROUPID
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Arg End: DXGROUPID: Value: 0.00;
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Execute Begin: _ORDERDIAGNOSES
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> Deleting row: 75
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> SQL Delete on object: _MasterOrdDx
12/17/2015 11:36:57.803 INFO Process Id #166652 ML Thread [Trace] 74166932> SQL Exec: DELETE FROM dbo.ORDDX WHERE ORDDXID=?
12/17/2015 11:36:57.804 INFO Process Id #166652 ML Thread [Trace] 74166932> SQL HostVar: 1 ORDDXID = 1765971243057310
12/17/2015 11:43:38.969 INFO Process Id #166652 Thread Id #1 [Trace] CMBCHub::AllViewsCanClose()

12/17/2015 11:43:38.969 INFO Process Id #166652 Thread Id #1 [Trace] CMBCHub::CanAllViewsClose, query ViewID = 1 - Centricity Practice Solution

Posted : December 18, 2015 12:31 pm

aballard

(@aballard)

Posts: 43

Trusted Member

We are running multi-server environment:

Physical Database Server box:
Windows Server 2008 R2 SQL Server 2008

JBoss Application Server:
Hyper-v host Windows 2008 R2

I think we are a little behind on a few windows updates.

Posted : December 18, 2015 12:54 pm

aballard

(@aballard)

Posts: 43

Trusted Member

Virtualization is running on Windows 2012 server.

Posted : December 18, 2015 12:56 pm

rchildress

(@rchildress)

Posts: 13

Active Member

Topic starter

Interesting. Did Centricity begin to respond again at 11:43:38?

Posted : December 18, 2015 1:03 pm

aballard

(@aballard)

Posts: 43

Trusted Member

That was the last activity in the db trace log before it froze.

Posted : December 18, 2015 1:12 pm

aballard

(@aballard)

Posts: 43

Trusted Member

Upon looking a little closer, the last two log entries may have been written after the client froze. We (me and our VAR) were looking at other stuff after the client froze for a few minutes so the last entries may have been 5 minutes later, with it coming back to life. Eventually it started working again randomly while I was working with VAR support. I didn't note the time it started working again for everybody, maybe 10-15 minutes later.

Posted : December 18, 2015 1:31 pm

BrianAPHN

(@brianaphn)

Posts: 106

Estimable Member

Have you noticed if it happens on the orders that have been auto generated by the Immunization Management HTML Form? We have intermittent orders that will freeze up for 3 to 5 minutes when they come from that form. It doesn't seem to matter if there are multiple immunization or just a single. We haven't been able to pin point a particular immunization either. If you toggle off the auto generate orders the issue no longer occurs.

Posted : December 21, 2015 4:37 am

jmeyersmd

(@jmeyersmd)

Posts: 12

Eminent Member

Hi Richard, et al,
We experienced the exact issue that Richard described: intermittent freezing of the client, usually when the order screen is showing, but unrelated to any particular action in that window. This brought our clinic essentially to a halt in November.

Our question : Did this occur for you at baseline SP11 with no patch, or with patches? If so, which ones?

The behavior appeared for us with SP10 Patch 3, not SP11. We backed off to SP10 (no patch) to resolve the problem. GE says it found the bug, related to window & scroll bar positions, and has a fix; but they also say that neither the SP10 patch 3 functionality that caused the problem, nor the fix, are in baseline SP11! So if this happened for you in baseline SP11 with no patch, then the bug GE found is NOT our bug - and that means my clinic is frozen at SP10 until it is found.

Thank you!
James Meyers, MD
Director of Medical Information Systems
Greater Lawrence Family Health Center

Posted : December 22, 2015 6:10 am

rchildress

(@rchildress)

Posts: 13

Active Member

Topic starter

Thank you for that update. The issue was first noticed at our practice on December 14, our first business day after upgrading to SP 11. No patches were applied at that time. The drug interactions and meaningful use patches were applied the next night, December 15. The issue continues to occur, seemingly at random. However, the occurrences that seemed to affect the entire organization at the exact same time have seemed to stop.

That being said...our users may have just become less sensitive to the issue and have stopped reporting it.

Posted : December 22, 2015 7:06 am

jmeyersmd

(@jmeyersmd)

Posts: 12

Eminent Member

Very helpful, thank you. Quick folow-up question:

- Did you upgrade to SP11 from SP10 (with no patch), or from what level?
- Did you see this bug in testing, if you use a test system? I ask because we have a full test system and cannot yet duplicate the problem there.

James Meyers

Posted : December 22, 2015 9:16 am

Welcome to the NEW Community Healthcare User Group ( the new CHUG)— your ultimate destination for insights and discussions at the intersection of healthcare and technology.

Please scroll to the bottom for “Centricity” related topics from the previous forum.

CPS 12.0.11 - Intermittent freezing while placing orders