Did you know you can write functions in C and then call them directly from Python? Isn't that cool? Let's skip all the background and the "why would I ever need to do this" for now and just dive on in to the code!
First, the C Function
To demonstrate, we're going to write a program in C to find the factorial of a number. If you don't remember factorials from high school, here's an example:
4! (read four factorial) = 4 * 3 * 2 * 1
That is what our C program is going to do. Fire up a text editor and lets crank this function out:
long factorial(int user_input) {
long return_val = 1;
if (user_input <= 0) {
return -1;
else {
for (long i = 1; i <= user_input; i++) {
return_val *= i;
}
}
return return_val;
}
int main() {
return 0;
}
We are defining a function called "factorial" which will return a "long." We're using long instead of int because factorial functions can return some pretty big numbers.
Next, we're declaring and initializing return_val which we'll use to return the value of the calculation.
Now, the if statement is ensuring the number passed in by the user is positive, and if not, to return the value of -1. We're returning -1 because later, when we wrap this function in Python, we're going to know that getting -1 back from the C function probably means there was bad input.
If the number returned is greater than 0, we enter our loop in which we use an iterator, i, and multiply our return_val variable by it until i is equal to the number passed in by the user. Basically, this loop is saying:
n! = 1 * 2 * 3 * 4 ... * n
The final part, with the int main() is to appease the C compiler when we turn this into a .so file. I may be mistaken, but I'm pretty sure this part is necessary even though it doesn't do anything. If anyone knows any better, please feel free to mention so.
The Pre-Python Part
Now that our C is written we have a couple things to do before we write the Python bit. First, save the .c file. I called mine cfactorial.c. Now, we have to turn this into a "shared object" file. In Linux, the command to do so is this:
$ cc -fPIC -shared -o cfactorial.so cfactorial.c
This particular command will make a cfactorial.so out of my cfactorial.c file. Now, to the actual Python
The Python Part
Almost done! Fire up that text editor again and lets script out some Python. First, we need to import the ctypes module. Then, if you're anything like me, you'll want to put the absolute path of the .so file into its own variable. So the top of my pyfactorial.py looks like this:
from ctypes import *
so_file = '/home/ewhiting/cstuff/cfactorial.so'
The next thing we want to do is create our cdll object out of our previously created .so file. So, after the so_file variable assignment, put:
cfactorial = CDLL(so_file)
Now, technically at this point you can start messing with calling the C function in the Python script by running python in the command line but lets be a little responsible first. Before we play with it some more, lets wrap our C function in a Python function. After creating the cfactorial variable, create the following function:
def factorial(num):
c_return = cfactorial.factorial(num)
if (c_return != -1):
return c_return
else:
return "C Function failed, check inputs"
Save this file as pyfactorial.py. Altogether, it should look like this:
from ctypes import *
so_file = '/home/ewhiting/cstuff/cfactorial.so'
cfactorial = CDLL(so_file)
def factorial(num):
c_return = cfactorial.factorial(num)
if (c_return != -1):
return c_return
else:
return "C Function failed, check inputs"
Note, the way to call functions inside the imported C shared object file is by saying <CDLL Object>.<function name from C code>(<parameter>). Easy!
So basically, any time we want to use that C function within Python, we call the factorial function which will run the C function with the parameter passed in by the user and evaluate the result. If the C function returns -1 (remember we put that in there), the Python script knows that there was a problem. Otherwise, it will return the number. Lets try it out! Fire up your terminal and start python
>>> import pyfactorial as pf
>>> pf.factorial(5)
120
>>> pf.factorial(10)
3628800
>>> pf.factorial(-4)
'C Function failed, check inputs'
Ta-da!! That's the basic idea behind using C functions in Python. This is definitely a tool worth having. Apply all your other programmerly knowledge to making awesome functions and features, and let me know if you have any questions.
Top comments (23)
Great example! Do you know if there is any tool to help you with using it in a package? So that if you install the package (using setup.py) it automatically compiles the C code and links paths to shared objects. Maybe even writes the wrapper function in Python. Something like Rcpp for R.
Hi Jan! Off the top of my head, I don't know of anything, but have a look at Fernando B (@kodaman2 ) comment a couple comments down. He mentions SIP that can turn C or C++ bindings. If you can't find the comment, this is the link he provided: pypi.org/project/SIP/
I dunno if sip is a fully automated solution, have a look at this project I believe you have to manually write the sip file (wrapper), but I could be wrong. I've never actually successfully compiled with sip.
github.com/dimv36/QCustomPlot-PyQt5
There is quite a few ways to interface Python with C/C++. I hadn't heard of SIP before, but I think SWIG and Shiboken should be fairly similar in creating bindings automagically.
Personally I'm a big fan of Cython and pybind11. Cython was created to be able to transpile a python-like syntax to C to then compile it, but it can also be used to wrap C libraries or to interface with C code.
Pybind11 is similar to BoostPython as in that is was meant to expose C functionality to Python similar to what is being done in this article, but it also helps with converting types such as lists and tuples and makes it easier to manipulate the GIL.
If you are interested in this topic I have a talk on embedding Python into C++. Pretty much the exact opposite of this :P
This is a really neat article, I didn't know it was this easy! Because I'm not familiar with the
ctypesmodule, andfrom ctypes import *makes it a little tough to see which functionality is coming fromctypes, is it just theCDLLclass that you're using from there?Yes! and thank you for pointing out the lack of clarity. the
CDLLis the only thing in the Python script coming from the ctypes moduleHi!, great example. It was very useful. I have one question... I don't know if is only me but when my number is 13 (or bigger) the code from C doesn't work. Anyone has the same problem?
it's probably because 13! is 6.2 billion and it's possible your compiler only sets aside enough memory for 4.2 billion for
long. trylong longand see if you get the same problemHi Erik, thanks for the reply. I did your recommendation but it doesn't work. I was thinking that the problem is with gcc but I am not sure. Do you have any other idea?
I tried it out too. It's definitely weird. It's "working" for me but it's not giving me the right answer. I never tested that high. I'm at work now and can't really dig into it but if I get some time I'll let you know.
Sure, thanks again!
Not sure if this thread will be active again, but I found that on the c side of the program (using long long) it is accurate up to 20! . I found this by adding a printf statement in cfactorial.c . So it seems that at some point in the process of python and c communicating, the true value is lost.
interesting. perhaps
ints in C are represented differently in memory than in Python. maybe the best way is to parse theintinto achararray and send it to Python as a string?That does sound promising because I imagine it could scale almost indefinitely if its not bound by the long or long long limits
noob question: Is it going to have the same performance as the performance of c programming language code.
the work that the C function is doing will be equally as fast. running The C function though the Python script will add a little latency because the Python script is doing a couple things before calling the function, but the difference in time should be negligible
One more question, is it going to be the same for c++ code?
I am not sure if this will work exactly the same with C++. the first difference that comes to mind is using
g++instead ofccthe ctypes library seems to be optimized for C specifically but I bet you can use c++ as well. I'm not in a place where I can test this out right now, but play around with it and let me know what you learn
Thank you so much. I will try it out and would share the results. Thanks once again, Mr. Erik.
Hi Erik,
Thanks for this nice explanation.
Can you also please share the example wherein the C program function will return char* and how to reinterpret this data in python code.
Thanks in advance.
Great read, have you used sip? pypi.org/project/SIP/ it can turn full c or c++ libs to python bindings. I've never had the need to use it, but looks promising.
interesting, I've not heard of this before. will check it out!
This is really helpful for times when you can't find python libraries to do what you need or if you need C because it runs faster. Thanks for sharing!