Login Page - Create Account

DTC Protocol Discussion Forum


Date/Time: Mon, 28 Nov 2022 02:23:13 +0000


Protocol buffers with golang - no response on wire to TRADE_ACCOUNTS_REQUEST

View Count: 1606

[2020-10-13 13:24:10]
vp_ftw - Posts: 12
Developing a DTC Client implementation in golang leveraging protocol buffers.

https://github.com/RileyR387/sc-dtc-client

Set Encoding - Works
Logon+Logon Respose - Works
Unsolicited balance updates - Works!
Auto Subscribe from server subs - Works
Auto Subscribed Market Data - Works

BUT:
I have not witnessed a heartbeat come across the wire from the server to the client, but my (client -> server heartbeat will keep the connection open all night parsing data)
I have been unable to get a response to "TRADE_ACCOUNTS_REQUEST" (Account List Messages)
I have been unable to get a response to "ACCOUNT_BALANCE_REQUEST" (Account Balance Data Messages)

I'm not sure where to go from here without seeing a working implementation... Any thoughts or advice would be appreciated!
[2020-10-16 02:44:05]
vp_ftw - Posts: 12
Here's what the message log looks like on the server side.

DTC Protocol server | Incoming connection from 127.0.0.1. | 2020-10-15 21:36:22.053
DTC client #3. 127.0.0.1 (4) | Creating socket. | 2020-10-15 21:36:22.062
DTC client #3. 127.0.0.1 (4) | New receive buffer size: 262144 | 2020-10-15 21:36:22.062
DTC client #3. 127.0.0.1 | Set DTC encoding to Protocol Buffers | 2020-10-15 21:36:22.067
DTC client #3. 127.0.0.1 | Requested heartbeat interval: 6 | 2020-10-15 21:36:22.087
DTC client #3. go-dtc. 127.0.0.1 | Received logon request. | 2020-10-15 21:36:22.087
DTC client #3. go-dtc. 127.0.0.1 | Using relay server mode. | 2020-10-15 21:36:22.088
DTC client #3. go-dtc. 127.0.0.1 | Sent successful Logon response message to client. Username: XXXXXXXX. Result text: Connected to SC DTC Protocol server. Service=cqg|SymbolSettings=cqg | 2020-10-15 21:36:22.088
DTC client #3. go-dtc. 127.0.0.1 | Sending all symbols to relay client. | 2020-10-15 21:36:24.074
DTC client #3. go-dtc. 127.0.0.1| Sent relay client initial market data messages for: F.US.M2KZ20. SymbolID: 1. ServiceCode: cqg | 2020-10-15 21:36:24.074
DTC client #3. go-dtc. 127.0.0.1| Sent relay client initial market data messages for: USDX. SymbolID: 2. ServiceCode: fxstatscalc | 2020-10-15 21:36:24.074
DTC client #3. go-dtc. 127.0.0.1| Sent relay client initial market data messages for: F.US.MESZ20. SymbolID: 3. ServiceCode: cqg | 2020-10-15 21:36:24.075
DTC client #3. go-dtc. 127.0.0.1| Sent relay client initial market data messages for: F.US.MNQZ20. SymbolID: 4. ServiceCode: cqg | 2020-10-15 21:36:24.075



Client side (my implementation):
time="2020-10-15 21:36:22.045" level=info msg="Connecting: XXXXXXX@127.0.0.1:11099\n" file=" connection.go:55"
time="2020-10-15 21:36:22.051" level=debug msg="Sending ENCODING_REQUEST" file=" connection.go:208"
time="2020-10-15 21:36:22.054" level=debug msg="Received ENCODING_RESPONSE(7) with bytes [8 0 0 0 4 0 0 0 68 84 67 0]" file=" connection.go:211"
{
"ProtocolVersion": 8,
"Username": "XXXXXXXXXXXXXXX",
"Password": "XXXXXXXXXXXXXXX",
"Integer1": 2,
"HeartbeatIntervalInSeconds": 6,
"TradeMode": "TRADE_MODE_LIVE",
"ClientName": "go-dtc"
}
time="2020-10-15 21:36:22.076" level=debug msg="Sending LOGON_REQUEST" file=" connection.go:161"
time="2020-10-15 21:36:22.079" level=debug msg="Received LOGON_RESPONSE result: Connected to SC DTC Protocol server. Service=cqg|SymbolSettings=cqg" file=" connection.go:180"
{
"ProtocolVersion": 8,
"Result": "LOGON_SUCCESS",
"ResultText": "Connected to SC DTC Protocol server. Service=cqg|SymbolSettings=cqg",
"ServerName": "SC DTC Server",
"TradingIsSupported": 1,
"OCOOrdersSupported": 1,
"OrderCancelReplaceSupported": 1,
"SecurityDefinitionsSupported": 1,
"MarketDepthIsSupported": 1,
"BracketOrdersSupported": 1,
"MarketDataSupported": 1
}
time="2020-10-15 21:36:22.091" level=info msg="Client listener started" file=" connection.go:42"
time="2020-10-15 21:36:24.073" level=info msg="Added security M2KZ20 from exchange cqg as F.US.M2KZ20" file=" securities.go:12"
time="2020-10-15 21:36:24.076" level=info msg="Added security USDX from exchange fxstatscalc as USDX" file=" securities.go:12"
time="2020-10-15 21:36:24.076" level=info msg="Added security MESZ20 from exchange cqg as F.US.MESZ20" file=" securities.go:12"
time="2020-10-15 21:36:24.078" level=info msg="Added security MNQZ20 from exchange cqg as F.US.MNQZ20" file=" securities.go:12"
(DTC_PB.TradeAccountsRequest, protoreflect.FullName)
{
"RequestID": 1
}
([8 1], []uint8)
time="2020-10-15 21:36:25.108" level=debug msg="Sending TRADE_ACCOUNTS_REQUEST" file=" accountData.go:29"
(DTC_PB.AccountBalanceRequest, protoreflect.FullName)
{
"RequestID": 2
}
([8 2], []uint8)
time="2020-10-15 21:36:28.115" level=debug msg="Sending ACCOUNT_BALANCE_REQUEST" file=" accountData.go:51"
time="2020-10-15 21:37:22.620" level=trace msg="Received ORDER_UPDATE(301)" file=" messageRouter.go:253"
Date Time Of Last Edit: 2020-10-16 02:52:12
[2020-10-19 04:00:41]
vp_ftw - Posts: 12
I've just upgraded from Sierra Chart 2177 to 2182 and am now receiving heartbeats from the internal DTC server!

This is progress, but at first blush there's still no response to TRADE_ACCOUNTS_REQUEST or ACCOUNT_BALANCE_REQUEST.

Thanks for the improvements! Hoping we can make further progress here.
[2020-10-21 22:20:56]
DTC Engineering - Posts: 320
This is progress, but at first blush there's still no response to TRADE_ACCOUNTS_REQUEST or ACCOUNT_BALANCE_REQUEST.
You should definitely get an immediate response to these even if there are no accounts available.
[2020-10-25 01:08:56]
vp_ftw - Posts: 12
That is what the docs state, but I'm not seeing one. I've verified bytes and message type byte order (LittleEndian) over the wire with wireshark and I can't see anything potentially wrong with my requests.

Is there a test case that proves these request/response flows are operational with protocol buffers?

I find it interesting that I didn't receive heartbeats from the server on SCv2177 but began to receive them after updating to v2182.

If protocol buffers aren't 100% complete and tested within the SC sever I'm happy to help there however I can.

I definitely prefer the ease of implementation of DTCProtol over FIX or CQG's web api. CQG's web api would also interfere with my human based trading requiring a second data subscription for API use! :P

Hoping we can get this figured out.

To illustrate my efforts on this side:
DEBUG [2020-10-24 19:17:13.100] accountData.go:28 Sending TRADE_ACCOUNTS_REQUEST
TRACE [2020-10-24 19:17:13.100] connection.go:260 Packed message with TypeID: 400 with length(6) and contents: (0x060090010801)

Expanding: 0x060090010801

// LittleEndian Message Length as uint16
06 00
// LittleEndian Message Type ID as uint16
90 01 // bigEndian 0x0190 == 400 == TRADE_ACCOUNTS_REQUEST
// 2 bytes of protopuf data
08 01

[2020-11-05 06:02:31]
DTC Engineering - Posts: 320
We apologize for the delay with the response. We do not always monitor this board every day.

Is there a test case that proves these request/response flows are operational with protocol buffers?
This can be verified by connecting Sierra Chart using the DTC Sub Instance service to an instance of Sierra Chart with the DTC Protocol Server enabled. The encoding method needs to be protocol buffers which is not controllable by users but we are able to control that and we did verify this does work. The accounts were sent.

But we do see that only the first response is responded to and any additional ones are ignored, if there are no new trade accounts. So we will fix that in the next release. This must be what you are experiencing.


I find it interesting that I didn't receive heartbeats from the server on SCv2177 but began to receive them after updating to v2182.
We do not know why this would be the case. There has been no change with heartbeats.


I definitely prefer the ease of implementation of DTCProtol over FIX or CQG's web api. CQG's web api would also interfere with my human based trading requiring a second data subscription for API use! :P
Wow. This is really good to hear. It is good to hear others, say what we definitely know to be true.
Date Time Of Last Edit: 2020-11-05 06:05:49

To post a message in this thread, you need to log in with your Sierra Chart account:

Login

Login Page - Create Account