The community forums are in BETA and closed to the public.

How to Import Your Legacy Data into PubNub Storage

How to import pre-existing messages from your legacy application into PubNub Storage

NOTE: Currently, this is only for the purpose of creating a batch process to migrate your data from your current legacy application (that does not use PubNub) to a new version of your application that does use PubNub so that you do not lose your historical message data after you migrate to the new version.

The ability to insert/update/delete messages in storage as a normal part of your application workflow will be possible in the near future (planned for November 2016). Until then, you can use a design pattern that provides this behavior.

Important Considerations

  • You have to contact PubNub Support to configure your keys to allow this behavior.
  • The ptto value must be a 17 digit PubNub timetoken that is not older than your Storage & Playback add-on's retention property. It is likely you require setting it to Unlimited since you are importing data for the purpose of not losing that history after the migration. Review this StackOverflow thread for converting a timestamp to a PubNub 17-digit timetoken.
  • If PubNub Access Manager add-on is enabled, you must use and auth_key with write permission for all channels you are populating. You could enable write permission for the entire key set during the duration of this batch process. Just perform the grant without a channel or an auth-key parameter with ttl = 0 (forever grant) before the process begins then revoke the permission after it finishes.

REST API

When you implement your batch process that performs this data migration, you will have to use REST calls as there are no current SDK APIs that support ptto or norep.

WRITE

This is just a publish that specifies the timetoken to use to store the message with the norep flag enabled. ~~~ http://pubsub.pubnub.com/publish/[your-publish-key]/[your-sub-key]/0/[channel]/0/[message]?ptto=[overriding-timetoken]&norep=1 ~~~

READ

This is just a standard history call for a single message using its publish timetoken. ~~~ http://pubsub.pubnub.com/v2/history/sub-key/[your-sub-key]/channel/[channel-name]?include_token=true&count=1&end=[publish-timetoken] ~~~

The timetoken is the message's unique id - 14184108487864105. When you publish new messages, the timetoken that is returned in the success callback is the timetoken used to retrieve that message from storage.


Comments to this discussion are now closed!