Load a Large spaCy model on AWS Lambda

Serverless NLP using spaCy and AWS Lambda

Jul 3 ·5min read

Paul Cézanne / Public domain from Wikimedia

spaCy is a useful tool that allows us to perform many natural language processing tasks. When integrating spaCy into an existing application, it is convenient to provide it as an API using AWS Lambda and API Gateway. However, due to Lambda’s limitations , it is hard to deploy large models.

In this article, I will show you how to deploy spaCy using the recently released feature to mount Elastic File System (EFS) on AWS Lambda . By using this feature, we can store a large size model to EFS and load it from Lambda functions. Specifically, it is possible to load data larger than the space available in Lambda’s /tmp (512MB).

The overall architecture is as follows. Lambda loads the spaCy package on Lambda Layers. EFS stores the spaCy models. Lambda then loads the models from EFS. For redundancy, four subnets are placed on two different availability zones.


  • Docker

Creating a VPC and subnets

First of all, we must configure VPC that can reach the EFS mount targets. Here, we create a VPC, an internet gateway, two NAT gateway, and four subnets(two public, two private).

In the VPC console, I select Create VPC and set a Name tag and IPv4 CIDR block as follows:

In the next step, I create an internet gateway to connect the Internet from the VPC with the following settings. After that, I attach it to the created VPC.

In the next step, I create four subnets( public-spacy-1, public-spacy-2, private-spacy-1, private-spacy-2 ) with the following settings. Notice that public-spacy-1 and private-spacy-1 have the same availability zone.

Then, I create two NAT gateways and attach it to public subnets to access the Internet from a lambda function in the private subnet. I gave NAT gateways names: NAT spaCy 1 and NAT spaCy 2 .

Finally, I create route tables. For public subnets, I add a destination and a target Internet Gateway to access the Internet. For private subnets, I add a destination and a target NAT spaCy 1 for private-spacy-1 and NAT spaCy 2 for private-spacy-2 .

Creating an Elastic File System

In the EFS console, I select Create file system and make sure that the default VPC and its subnets are selected. For all subnets, I use a security group that allows network access to other resources in the VPC. For simplicity, I set up a security group that allows all traffic.