Snakemake input functions
Snakemake input functions.
We will start with rule cat, which takes two files and makes a new one named all.
What cat does is take the content of the input files and stick them together in the output files.
Within the files sample_a and sample _b, we have a short declarative string.
rule cat:
input: "sample_a.txt", "sample_b.txt"
output: "all.txt"
shell: "cat {input} x {output}"
We launch the pipeline, rule cat is run, and file all is generated.
If we open the file all we see is a concatenation of the two input files.
########
So if we look at the input, we see that both files are the same except for one letter. Since this is a common occurrence, there is a built-in function in Snakemake to deal with these situations named expand.
We can use it like this.
rule copy:
input: expand("sample_{x}.txt", x =["a","b"])
output: "all.txt"
shell: "cat {input} x {output}"
When we launch the pipeline, the rule is run, and we can see that the intended shell command is generated as well as the all file.
########
An important aspect of expand is that it can take wildcards propagating from the output.
Let's make the extension of the file a wildcard, and let's name it y.
If you are not sure how wildcards work, I will encourage you to check the videos from this series on the topic. Also, we have a video explaining rule all.
rule all:
input: "all.txt"
#
rule copy:
input: expand("sample_{x}.{y}", x =["a","b"])
output: "all.{y}"
shell: "cat {input} x {output}"
When we launch the pipeline, it crashes with a wildcard error claiming that no values were given for wildcard y.
WildcardError in file /Users/Marcos/Desktop/snakemake/Snakefile, line 5:
No values given for wildcard 'y'.
File "/Users/Marcos/Desktop/snakemake/Snakefile", line 5, in module
#########
What we need to do is indicate with a double curly bracket that the y in the input is actually the wildcard associated with the y wildcard from the input.
rule all:
input: "all.txt"
#
rule copy:
input: expand("sample_{x}.{{y}}", x=["a","b"])
output: "all.{y}"
shell: "cat {input} x {output}"
Now the pipeline launches, the rules are run, the shell string is well formed, and we get the all.txt file.
########
The input also admits native python functions, such as lambda. So we can write the equivalent string of input files.
Note that here we have to give wildcards as a parameter of the function.
rule all:
input: "all.txt"
#
rule copy:
input:
lambda wildcards: \
["sample_" + x + "." + wildcards.y \
for x in ["a", "b"]]
output: "all.{y}"
shell: "cat {input} x {output}"
We can launch the pipeline, the rules are run, the shell command is well formed, and we get the all.txt file.
##############
Actually, we can pass any function to the input to customize it as much as we want. For example, we can write an equivalent function using a for loop.
So now we need to specify that the rule input should be the function we just wrote.
rule all:
input: "all.txt"
#
X = ["a", "b"]
def the_files(wildcards):
files=[]
for letter in X:
file = "sample_" + letter + "." + wildcards.y
files = files+[file]
return files
#
rule copy:
input: the_files
output: "all.{y}"
shell: "cat {input} x {output}"
Again the pipeline does not crash, the shell command is well formed, and we get the all.txt file as well.