Amazon SQS Simple Queue Service is an alternative queue server also developed by Amazon and is also part of the AWS service infrastructure. If you prefer to use the "cloud" option, you can use the prebuilt documentation for its installation.
In addition to the ability to use this queue server within AWS, you can also use the can also use the local installation of this system on your own servers. If you prefer this option, you can use the softwaremill's implementation of the Amazon SQS server.
After you have created the SQS server, you need to specify the following
connection settings in the
sqs configuration settings. Unlike AMQP and Beanstalk,
SQS requires more values to set up a connection and will be different from what we are used to.
we're used to.
sqs: # Required AccessKey ID. # Default: empty key: access-key # Required secret access key. # Default: empty secret: api-secret # Required AWS region. # Default: empty region: us-west-1 # Required AWS session token. # Default: empty session_token: test # Required AWS SQS endpoint to connect. # Default: http://127.0.0.1:9324 endpoint: http://127.0.0.1:9324
Please note that although each of the sections contains default values, it is marked as "required". This means that in almost all cases they are required to be specified in order to correctly configure the driver.
After you have configured the connection - you should configure the queue that will use this connection:
You may also skip the whole
sqsconfiguration section (global, not the pipeline) to use the AWS IAM credentials if the RR is inside the EC2 machine. RR will try to detect env automatically by making a http request to the
http://169.254.169.254/latest/dynamic/instance-identity/as pointer here
version: "3" sqs: # SQS connection configuration... jobs: pipelines: test-sqs-pipeline: # Required section. # Should be "sqs" for the Amazon SQS driver. driver: sqs config: # Optional section. # Default: 10 prefetch: 10 # Consume any payload type (not only Jobs structured) # Default: false consume_all: false # Get queue URL only # Default: false skip_queue_declaration: false # Optional section. # Default: 0 visibility_timeout: 0 # Optional section. # Default: 0 wait_time_seconds: 0 # Optional section. # Default: default queue: default # Optional section. # Default: empty attributes: DelaySeconds: 42 # etc... see https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SetQueueAttributes.html # Optional section. # Default: empty tags: test: "tag"
Here is a detailed description of each of the SQS-specific options:
prefetch - Number of jobs to prefetch from the SQS until ACK/NACK.
visibility_timeout - The duration (in seconds) that the received messages are hidden from subsequent retrieve requests
after being retrieved by a
ReceiveMessage request. Max value is
43200 seconds (12 hours). Default:
wait_time_seconds - The duration (in seconds) for which the call waits for a message to arrive in the queue before
returning. If a message is available, the call returns sooner than WaitTimeSeconds. If no messages are available and the
wait time expires, the call returns successfully with an empty list of messages.
queue - SQS internal queue name. Can contain alphanumeric characters, hyphens (
-), and underscores (
Default value is
skip_queue_declaration - By default, RR tries to declare the queue by default and then gets the queue URL. Set this
true if the user already declared the queue to only get its URL.
consume_all - By default, RR supports only
Jobs structures from the queue. Set this option to true if you want to
also consume the raw payloads.
attributes - List of
the AWS SQS attributes.
attributes: DelaySeconds: 0 MaximumMessageSize: 262144 MessageRetentionPeriod: 345600 ReceiveMessageWaitTimeSeconds: 0 VisibilityTimeout: 30
tags - Tags don't have any semantic meaning. Amazon SQS interprets tags as character.
This functionality is rarely used and slows down the work of queues: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-queue-tags.html