Showing posts with label broker. Show all posts
Showing posts with label broker. Show all posts

Saturday, February 25, 2012

Messages stuck in sys.transmission_queue

Hi,

At my company, we're trying to use service broker to create a client-server system where there is a head office machine and multiple outlets registered with that head office. My problem is that sometimes when a branch sends a message to the head office, it just seems to sit in the transmission queue and never gets sent. If I run a script that forcibly ends the conversations on the client machine (with cleanup), storing the message bodies and then resend them, they seem to get through fine.

The way that we send messages is by calling a t-sql stored procedure from a c# application using SqlCommand (don't know if this should make any difference).

If I monitor the Head Office machine and one of the Outlets while this is happening, on the HO I get three events in a row:

Broker: Message Classify (1 - Local) Audit Broker Conversation (2 - No Certificate) Broker: Message Undeliverable (1 - Sequenced Message)
The TextData contained in the third event is: This message could not be delivered because the security context could not be retrieved.

The RoleName of the server is Initiator, and the TargetUserName is the name of the service on the Outlet.

On the Outlet I get the following event repeatedly (presumably as it continues to try sending the message) - Broker: Remote Message Acknowledgement (1 - Message With Acknowledgement Sent).

On the client the RoleName also appears to be Initiator, and the TargetUserName is blank.

This would make me suspect that certificates were missing or something, except that if I remove messages from the queue and resend them they seem to get through, and also I've checked both databases and they have the correct certificates.

Any ideas?

Thanks in advance,

AdamThanks for the detailed info, it is always helpfull to have the full details when trying to diagnose an issue.
The most likely culprit in such cases is an account that has more certificates to be picked from when encrypting/signing a message. Whenever SSB has more than one certificates 'valid' to be used for a particular database principal, it will pick the one with the latest expiration date. So sometimes even though the configuration 'seems' correct, SSB will pick the 'wrong' certificate simply because it has multiple choices. The typical account prone to this problems is 'dbo', as certs are being created for dbo for various reasons and by default they are all valid for SSB to pick.
To confirm this problem, the 2nd event in your post (the Audit Broker Conversation one) actually tells which certificate was not found: it has the issuer name and serial number of the cert 'not found' on two of the columns (I can't remember now which ones and the BOL 'ommits' this detail). You can then look into the message sender's database to see why that particular cert was picked.
Certs can actualy be controled whether they are available for SSB to 'pick' by turning off the ACTIVE_FOR_BEGIN_DIALOG option on them (the option refers to certs used for the reverse path from target to initiator as well, despite the name 'begin_dalog'...)|||Thanks for the reply. What you're saying makes sense, and seems possible to me (we re-registered one of the outlets recently which could have caused this). The only problem is that the Audit Broker Conversation event doesn't appear to have the issuer or serial number in any of the columns. Maybe I'm using profiler wrong or something, but I would have thought that just selecting the event would have been enough. Any idea what I'm doing wrong?

Adam

|||

IssuerName is DBUserName and SerialNumber is in TargetLoginName.

Make sure the columns above are selected when defining the trace.

|||I had worked out that the IssuerName was DBUserName, but hadn't noticed the serial number. Thanks for explaining that. The thing is that the IssuerName isn't much use to me because the way I generated the certificates, they all have the same name Smile.

But anyhow, I found a solution to the problem. Basically we had an old database on the same server that had broker enabled as well, and I'm guessing that since they both have the same service names in them, that was causing the problems. I disabled service broker on the old db and everything seems to be working again. The only thing I'm worried about now is how the broker became enabled on the database - but that's not really a service broker issue.

Thanks for the help.

Adam
|||

Whenever SSB finds multiple instances of the same service it assumes a load balancing scenario and will hash the incomming conversations to the available service instances. Specifying a broker_instance in the begin dialog from the initiator would make the application stick to a specific service (database), even if more instances of the service show up on the target.

Messages stopped appearing in Queue!

I recently restored my database from a backup and it seems that something in my service broker setup has stopped working, no messages now appear.

I checked that the broker was enabled (it wasnt and i had to use NEW_BROKER because of the 'same id' message)

The database is still set to trustyworthy.

There are also no messages in sys.transmission_queue

I used profiler with all the broker events enabled and the result was as follows:

Broker:Conversation Group
Broker:Conversation STARTED_OUTBOUND

Broker:Conversation CONVERSING

Broker:Message Classify
Broker:Conversation CLOSED
Broker:Conversation Group
Broker:Message Classify

But i'm not sure whether this highlights a problem or not? Any help gratefully appreciated!!

Thanks,

Andy

Hi,

I had the complete conversation - including the end of the conversation - in a stored procedure.

This was the reason i wasn't getting anything in sys.transmission_queue - i didnt end the conversation and the error appeared.

It turned out to be a permissions thing as when i'd restored the database it had set the owner to [SA]

Sorry for wasting your time if you'd bothered to read my problem!

Cheers,

Andy

Messages stopped appearing in Queue!

I recently restored my database from a backup and it seems that something in my service broker setup has stopped working, no messages now appear.

I checked that the broker was enabled (it wasnt and i had to use NEW_BROKER because of the 'same id' message)

The database is still set to trustyworthy.

There are also no messages in sys.transmission_queue

I used profiler with all the broker events enabled and the result was as follows:

Broker:Conversation Group
Broker:Conversation STARTED_OUTBOUND

Broker:Conversation CONVERSING

Broker:Message Classify
Broker:Conversation CLOSED
Broker:Conversation Group
Broker:Message Classify

But i'm not sure whether this highlights a problem or not? Any help gratefully appreciated!!

Thanks,

Andy

Hi,

I had the complete conversation - including the end of the conversation - in a stored procedure.

This was the reason i wasn't getting anything in sys.transmission_queue - i didnt end the conversation and the error appeared.

It turned out to be a permissions thing as when i'd restored the database it had set the owner to [SA]

Sorry for wasting your time if you'd bothered to read my problem!

Cheers,

Andy

Message size limitations on Service Broker?

Hi Everyone
Does anyone know if there is/isn't a size limitation on the Service
Broker. I have a remote machine running with express 2005 and I would
like to send message to/from the machine in Head Office running SQL
Server Enteprise 2005.
Thanks in advance
EricThe max message size of the service broker is 2 gigabytes (one billion
Unicode characters) if I am not mistaken.
--
/*
Warren Brunk - MCITP - SQL 2005, MCDBA
www.techintsolutions.com
*/
<ricolee99@.gmail.com> wrote in message
news:1158184604.504436.77770@.m73g2000cwd.googlegroups.com...
> Hi Everyone
> Does anyone know if there is/isn't a size limitation on the Service
> Broker. I have a remote machine running with express 2005 and I would
> like to send message to/from the machine in Head Office running SQL
> Server Enteprise 2005.
> Thanks in advance
> Eric
>

Message size limitations on Service Broker?

Hi Everyone
Does anyone know if there is/isn't a size limitation on the Service
Broker. I have a remote machine running with express 2005 and I would
like to send message to/from the machine in Head Office running SQL
Server Enteprise 2005.
Thanks in advance
Eric
The max message size of the service broker is 2 gigabytes (one billion
Unicode characters) if I am not mistaken.
/*
Warren Brunk - MCITP - SQL 2005, MCDBA
www.techintsolutions.com
*/
<ricolee99@.gmail.com> wrote in message
news:1158184604.504436.77770@.m73g2000cwd.googlegro ups.com...
> Hi Everyone
> Does anyone know if there is/isn't a size limitation on the Service
> Broker. I have a remote machine running with express 2005 and I would
> like to send message to/from the machine in Head Office running SQL
> Server Enteprise 2005.
> Thanks in advance
> Eric
>

Message size limitations on Service Broker?

Hi Everyone
Does anyone know if there is/isn't a size limitation on the Service
Broker. I have a remote machine running with express 2005 and I would
like to send message to/from the machine in Head Office running SQL
Server Enteprise 2005.
Thanks in advance
EricThe max message size of the service broker is 2 gigabytes (one billion
Unicode characters) if I am not mistaken.
/*
Warren Brunk - MCITP - SQL 2005, MCDBA
www.techintsolutions.com
*/
<ricolee99@.gmail.com> wrote in message
news:1158184604.504436.77770@.m73g2000cwd.googlegroups.com...
> Hi Everyone
> Does anyone know if there is/isn't a size limitation on the Service
> Broker. I have a remote machine running with express 2005 and I would
> like to send message to/from the machine in Head Office running SQL
> Server Enteprise 2005.
> Thanks in advance
> Eric
>

Message sent retry timeout

Hi

here is from BOL

>When a conversation is marked delayed, Service Broker performs the matching process again after a timeout period. Notice that failure to find a matching route is not considered an error.

for example it could happen when there is no route for a service.
the question is - what is this timeout. Is it configurable? on a system wide
basis ?

It will be great if this process is described in more details.

Best regards,

Leonid.The timeout period is not configurable. If you do want to force reclassification, however, you could trigger that by running ALTER ROUTE on some pre-existing route that basically does not change any attributes. The ALTER ROUTE would then have no effect, but it would trigger metadata change which causes Service Broker to retry classifying delayed conversations.|||

But what is the timeout is ? What is the overhead when SB trying to connect to a sevice with no route. As I understood SB will try to send a message every minute. Could you please clarify.

Leonid.

|||4 seconds that exponentially backs out to about a minute. But don't quote me on this.

Monday, February 20, 2012

message lost

Hi...

My service broker was working...all of a sudden it stopped working...so i figured out destination queue was disabled and I set it as

ALTER QUEUE SERVERQUEUE

WITH STATUS = ON, RETENTION = ON, ACTIVATION (

STATUS = ON, PROCEDURE_NAME = usp_OnReceiveMessage,

MAX_QUEUE_READERS = 5, EXECUTE AS OWNER )

Also altered my database for trustworthy after going through some posts previously.

Now the message is sent...i dont see any message in my tranmission queue but i dont receive the message in destination server.

When i checked endpoints...select * from sys.conversation_endpoints I see the last message with st_desc as conversing...

how do I solve this...?

Thanks,

Pramod

The queue must have been disabled because you must have rolled back a transaction that performed a RECEIVE on the queue 5 or more times consecutively. Read about poised message handling here.

If the message is neither in transmission queue nor in the destination queue, chances are that the dialog was errored. Look in the initiator queue as well. I don't get what you mean by "the last message" in sys.conversation_endpoints. The conversation endpoints view does not contain any messages. The st_desc column describes the state of the conversation endpoint.

|||

Rushi:

I reconfigured the service broker...this time...in transmission queue its showing this error....

Connection handshake failed. The certificate used by the peer is invalid due to the following reason: The database principal has no mapping to a server principal.
State 89.

What does this mean ?

Thanks,

|||

You are using certificates for adjacent (i.e. transport) security. The certificate used by the peer belongs to a user in master who is not mapped to a login. This could happen either because:

i) You created a user in master without login.

ii) You created a user in master with a login, but later dropped that login, thus orphaning that user.

The fix would be to create a SQL login, then create a user in master for this login and alter authorization of the peers certificate to be owned by this user. Next grant connection permission on the service broker endpoint to the newly created login.

Hope that helps.

|||

Rushi:

I want to confirm few things here...before proceeding...

Lets take a login sbuser(created to login with sql authentication). This is the login i would be using to create endpoints, certificates, service etc for server service broker.

Should I have to login as sa to create those or I am fine with sbuser ?

Next step...in master database I create a user lets say proxysbuser which will be used to connect endpoints on both sides ie.initiator and target service brokers.

Then I i use proxysbuser to alter authorization of the peers certificate.

Am I in the right track ? Let me know.

Thanks,

Pramod

|||

Configuring dialog security (i.e. between services) is independent of configuring adjacent security (i.e. between instances) and should not be confused. For configuring adjacent security the steps are as follows:

1> Instance A: Create certificate owned by the dbo of master.
2> Instance A: Create SSB endpoint using the above certificate for authentication.
3> Instance A: Export certificate to \\shared\A.crt

4> Instance B: Create certificate owned by the dbo of master.
5> Instance B: Create SSB endpoint using the above certificate for authentication.
6> Instance B: Export certificate to \\shared\B.crt

7> Instance A: Create login for instance B, let's say InstanceB.
8> Instance A: Create user in master for login InstanceB.
9> Instance A: Import certificate \\shared\B.crt into master with owner InstanceB.
10> Instance A: Grant connect permission on SSB endpoint to login InstanceB.

11> Instance B: Create login for instance A, let's say InstanceA.
12> Instance B: Create user in master for login InstanceA.
13> Instance B: Import certificate \\shared\A.crt into master with owner InstanceA.
14> Instance B: Grant connect permission on SSB endpoint to login InstanceA.

The steps for securing dialogs are different from this. Refer to the docs for that.

If you find this tedious, you can use Remus' Service Listings Manager which automates this process for you.

Rushi

|||

Rushi:

I have been trying to use Remus Service Listing I have been refering to this link...https://blogs.msdn.com/remusrusanu/archive/2006/04/07/571066.aspx

In this step The administrator of the SQL Server instance hosting service ‘A’

In import mode the created script throws this error....

Msg 6522, Level 16, State 1, Procedure ssb_create_certificate_from_blob, Line 0 A .NET Framework error occurred during execution of user defined routine or aggregate 'ssb_create_certificate_from_blob': System.Data.SqlClient.SqlException: A certificate with name 'COSCOSB' already exists or this certificate already has been added to the database.
System.Data.SqlClient.SqlException:
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnectionSmi.EventSink.ProcessMessagesAndThrow(Boolean ignoreNonFatalMessages)
at Microsoft.SqlServer.Server.SmiEventSink_Default.ProcessMessagesAndThrow(Boolean ignoreNonFatalMessages)
at System.Data.SqlClient.SqlCommand.RunExecuteNonQuerySmi(Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Microsoft.SqlServiceBroker.Samples.CertificatesBlob.CertificatesBlob.ssb_create_certificate_from_blob(SqlString databaseName, SqlString certificateName, SqlString authorizationUser, SqlBinary certificateBody)

The script which is throwing error is IF EXISTS(SELECT * FROM sys.certificates WHERE name = 'COSCOSB')
DROP CERTIFICATE [COSCOSB];
declare @.dbname sysname;
select @.dbname = DB_NAME();
print @.dbname -- this prints master
exec msdb.dbo.ssb_create_certificate_from_blob @.dbname, 'COSCOSB','COSCOSB',0x308201ED30820156A00302010202104B7566A1DEA72E844040F300FC947B6C300D06092A864886F70D010105050030353133303106035504031E2A0043004F00530043004F005F00430045005200540049004600490043004100540045005F005300550042301E170D3036303832373030303030305A170D3037303832383030303030305A30353133303106035504031E2A0043004F00530043004F005F00430045005200540049004600490043004100540045005F00530055004230819F300D06092A864886F70D010101050003818D0030818902818100D5DDA3F09C737A7F39E9E262C6B6C62524C3C8368750DC014AEE582FCDF97E29760C0452513414520FD8DB0A8CD3A389E15D0C5C5B006228145E405ADC2344ABF0DC08BF7647BEE413525C5E29B39107424479D70F25CD32F4FB95B43974E785C81410EF506EEC59E2C6CC6135F009300F5B496FF837E50AF8EB02F806CA99870203010001300D06092A864886F70D01010505000381810038C506E25FB74C59D08C58E56D9ED505A01AE0C33E5F28E5135D9E492F5E6B33703550E14042FF87EE83E8F8C43522EAEE9EDA417E7178BFB8BF7B5B27EA05926D4C52C9CAE9B7915A890422FCE67A4ED473E29154E1205A3CAF3245B0477A469192D409A0C5EB52BC307217CF7CA3C675F7305FC11503ECA17503EB1B493A4D;
GO

I dont have any certifcate in sys.certificates....why is it throwing error ? Could you pls tell me.

Thanks,
Pramod

.

|||

I think this is a known bug. I'm not sure if Remus has fixed it yet. You can try using the latest version and see if that works. Presently, Remus is on vacation, but I'll ask him to follow up on this bug when he gets back.

In the mean time, look at sys.certificates for any other certificate which may have been created (possibly with a different name).

Rushi

|||

Rushi:

Thanks for quick response...but I am sure i am using the latest version Service Lsiting Manager v1.1.3 which says that bug is fixed...sys.certifcates is empty. But will keep working on it.

Thanks,

Pramod

|||Are you in the right database?|||

Yup....I am in master....

|||

Rushi...my client db is standard sql2005 and server is developer sql2005 do u think that might be causing problem ?

|||No. The error is unrelated to licensing.|||On the correct instance? Service Listing Manager is simply running the SELECT * FROM sys.certificates query and finding a certificate but you are not.|||

Rushi:

Service Listing Manager is working fine in developer edition...I tried with 2 instances of standard edition on 2 different m/c it fails at this step exec msdb.dbo.ssb_create_certificate_from_blob @.dbname .......

message could not be delivered errors in service broker

Hi,

I am using service broker in between two database servers. following is the way i am sending and receiving messages

Send

BEGIN TRAN
BEGIN DIALOG CONVERSATION @.handle
FROM SERVICE @.SendService
TO SERVICE @.ReceiveService
ON CONTRACT @.Contract
WITH LIFETIME = @.lifetime;

SEND ON CONVERSATION @.handle
MESSAGE TYPE @.xmlMessageType(@.xmlMessage);
COMMIT

Receive

BEGIN TRAN;
RECEIVE TOP(1) @.xmlMessage = message_body,
@.handle = conversation_handle,
@.message_type = message_type_name
FROM TransactionQueue;

-
-- Check to make sure a message was returned to process. In theory this should never happen.
-
IF @.@.rowcount = 1
BEGIN

IF @.message_type = 'http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog'
BEGIN
END CONVERSATION @.handle;
COMMIT
RETURN 0
END

IF @.message_type = N'http://schemas.microsoft.com/SQL/ServiceBroker/Error'
BEGIN
RAISERROR(N'Received error %s from service [Target]', 10, 1)
END CONVERSATION @.handle;
COMMIT
RETURN 0
END


SET @.sql = 'EXEC '+@.message_processor_name+' @.xml'

BEGIN TRAN
EXEC sp_executesql @.sql, N'@.xml XML', @.xml=@.xmlMessage
COMMIT TRAN
END CONVERSATION @.handle;
END
COMMIT

I see Messages are delivered to the target every thing working fine other than following errors which i am seeing in profiler.

1) "This message could not be delivered because the conversation endpoint has already been closed." I see this error on initiator end. Is it like ending conversation on initiator end when i get "EndDialog" send an acknowledgement, which cannot be recieved by target as it has already ended conversation.

2) "An error occurred while receiving data: '64(The specified network name is no longer available.)'." I don't have much idea about the reason for this error. But in profiler i see value for GUID is different for this error and the real message.

Let me know if you need any other information


Please let me know if I have missed anything in above post|||

Just wondering if you resolved this issue as I am having the same issues with very similar code.

If you have resolved this, could you share what you did please so that I can (hopefully) resolve my problems too?

Thanks in advance.

message could not be delivered errors in service broker

Hi,

I am using service broker in between two database servers. following is the way i am sending and receiving messages

Send

BEGIN TRAN
BEGIN DIALOG CONVERSATION @.handle
FROM SERVICE @.SendService
TO SERVICE @.ReceiveService
ON CONTRACT @.Contract
WITH LIFETIME = @.lifetime;

SEND ON CONVERSATION @.handle
MESSAGE TYPE @.xmlMessageType(@.xmlMessage);
COMMIT

Receive

BEGIN TRAN;
RECEIVE TOP(1) @.xmlMessage = message_body,
@.handle = conversation_handle,
@.message_type = message_type_name
FROM TransactionQueue;

-
-- Check to make sure a message was returned to process. In theory this should never happen.
-
IF @.@.rowcount = 1
BEGIN

IF @.message_type = 'http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog'
BEGIN
END CONVERSATION @.handle;
COMMIT
RETURN 0
END

IF @.message_type = N'http://schemas.microsoft.com/SQL/ServiceBroker/Error'
BEGIN
RAISERROR(N'Received error %s from service [Target]', 10, 1)
END CONVERSATION @.handle;
COMMIT
RETURN 0
END


SET @.sql = 'EXEC '+@.message_processor_name+' @.xml'

BEGIN TRAN
EXEC sp_executesql @.sql, N'@.xml XML', @.xml=@.xmlMessage
COMMIT TRAN
END CONVERSATION @.handle;
END
COMMIT

I see Messages are delivered to the target every thing working fine other than following errors which i am seeing in profiler.

1) "This message could not be delivered because the conversation endpoint has already been closed." I see this error on initiator end. Is it like ending conversation on initiator end when i get "EndDialog" send an acknowledgement, which cannot be recieved by target as it has already ended conversation.

2) "An error occurred while receiving data: '64(The specified network name is no longer available.)'." I don't have much idea about the reason for this error. But in profiler i see value for GUID is different for this error and the real message.

Let me know if you need any other information


Please let me know if I have missed anything in above post|||

Just wondering if you resolved this issue as I am having the same issues with very similar code.

If you have resolved this, could you share what you did please so that I can (hopefully) resolve my problems too?

Thanks in advance.