pedro
/
django-orchestra-test
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

initial web tests

This commit is contained in:
Marc 2014-10-10 14:39:46 +00:00
parent b7758c97a5
commit 10e19fcdb4
26 changed files with 403 additions and 128 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
ROADMAP.md
View File

@ -13,8 +13,8 @@ Note `*` _for sustancial progress_
2. [x] [Orchestra-orm](https://github.com/glic3rinu/orchestra-orm) a Python library for easily interacting with Orchestra REST API
3. [x] Service orchestration framework
4. [ ] Data model, crazy input validation, admin and REST interfaces, permissions, unit and functional tests, service management, migration scripts and documentation of:
1. [x] PHP/static Web applications
1. [x] Websites with Apache
1. [ ] *PHP/static Web applications
1. [ ] *Websites with Apache
2. [x] FTP/rsync/scp/shell system accounts
2. [ ] *Databases and database users with MySQL
1. [x] Mail accounts, aliases, forwards with Postfix and Dovecot

42
TODO.md
View File

@ -16,9 +16,7 @@ TODO
* move invoice contact to invoices app?
* PHPbBckendMiixin with get_php_ini
* Apache: `IncludeOptional /etc/apache2/extra-vhos[t]/account-site-custom.con[f]`
* rename account.user to main_user
* webmail identities and addresses
* cached -> cached_property
* user.roles.mailbox its awful when combined with addresses:
* address.mailboxes filter by account is crap in admin and api
* address.mailboxes api needs a mailbox object endpoint (not nested user)
@ -50,16 +48,13 @@ Remember that, as always with QuerySets, any subsequent chained methods which im
* EMAIL backend operations which contain stderr messages (because under certain failures status code is still 0)
* Settings dictionary like DRF2 in order to better override large settings like WEBSITES_APPLICATIONS.etc
* DOCUMENT: orchestration.middleware: we need to know when an operation starts and ends in order to perform bulk server updates and also to wait for related objects to be saved (base object is saved first and then related)
orders.signales: we perform changes right away because data model state can change under monitoring and other periodik task, and we should keep orders consistency under any situation.
dependency collector with max_recursion that matches the number of dots on service.match and service.metric
* backend logs with hal logo
* Use logs for storing monitored values
* set_password orchestration method?
@ -94,9 +89,8 @@ Remember that, as always with QuerySets, any subsequent chained methods which im
return order.register_at.date()
* mail backend related_models = ('resources__content_type') ??
* ignore orders
* Dropdown menu for Account services/management object-tools
* ignore orders (mark orders as ignored)
* Domain backend PowerDNS Bind validation support?
@ -108,37 +102,23 @@ Remember that, as always with QuerySets, any subsequent chained methods which im
*jabber with mailbox accounts (dovecto mail notification)
* rename accounts register to manager register
* make accounts django auth users
- when an account is created a mirrored system user is created
- system users are independent users, so they can have different passwords and all.
* rename accounts register to manager register or accounttools, accountutils
* take a look icons from ajenti ;)
* Disable services is_active should be computed on the fly in order to distinguish account.is_active from service.is_active when reactivation.
* Perhaps it is time to create a ServiceModel ?
* COpy account.main_user.username to account.name for performance
* service backend execution dependency? first create user on NIS master then create directories on service server
* prevent deletion of main user by the user itself
* AccountAdminMixin auto adds 'account__name' on searchfields and handle account_link on fieldsets
* Separate panel from server passwords? Store passwords on panel? set_password special backend operation?
* be more explicit about which backends are resources and which are service handling
* What fields we really need on contacts? name email phone and what more?
* Redirect junk emails and delete every 30 days?
* DOC: Complitely decouples scripts execution, billing, service definition
@ -149,24 +129,12 @@ Remember that, as always with QuerySets, any subsequent chained methods which im
* Unify all users
* backend admin message with link
* delete main user -> delete account or prevent delete main user
APPS app?
* https://blog.flameeyes.eu/2011/01/mostly-unknown-openssh-tricks
* Ansible orchestration *method* (methods.py)
* interdependency user <-> account with the old usermodel
* pip upgrade or install
* disable account triggers save on cascade to execute backends save(update_field=[])
* validate database user names
* multiple domains creation; line separated domains
* Move MU webapps to SaaS?
* DN: Transaction atomicity and backend failure

7
orchestra/apps/accounts/models.py
View File

@ -49,6 +49,10 @@ class Account(auth.AbstractBaseUser):
def is_staff(self):
return self.is_superuser
@property
def main_systemuser(self):
return self.systemusers.get(is_main=True)
@classmethod
def get_main(cls):
return cls.objects.get(pk=settings.ACCOUNTS_MAIN_PK)
@ -63,7 +67,8 @@ class Account(auth.AbstractBaseUser):
created = not self.pk
super(Account, self).save(*args, **kwargs)
if created and hasattr(self, 'systemusers'):
self.systemusers.create_user(self.username, account=self, password=self.password, is_main=True)
self.systemusers.create(username=self.username, account=self,
password=self.password, is_main=True)
def disable(self):
self.is_active = False

2
orchestra/apps/domains/backends.py
View File

@ -15,6 +15,7 @@ class Bind9MasterDomainBackend(ServiceController):
('domains.Record', 'domain__origin'),
('domains.Domain', 'origin'),
)
ignore_fields = ['serial']
@classmethod
def is_main(cls, obj):
@ -25,6 +26,7 @@ class Bind9MasterDomainBackend(ServiceController):
def save(self, domain):
context = self.get_context(domain)
domain.refresh_serial()
print domain.render_zone()
context['zone'] = ';; %(banner)s\n' % context
context['zone'] += domain.render_zone()
self.append("{ echo -e '%(zone)s' | diff -N -I'^;;' %(zone_path)s - ; } ||"

1
orchestra/apps/domains/models.py
View File

@ -1,5 +1,4 @@
from django.db import models
from django.utils.functional import cached_property
from django.utils.translation import ugettext_lazy as _
from orchestra.core import services

4
orchestra/apps/domains/serializers.py
View File

@ -34,6 +34,8 @@ class DomainSerializer(AccountSerializerMixin, serializers.HyperlinkedModelSeria
try:
validators.validate_zone(domain.render_zone())
except ValidationError as err:
self._errors = { 'all': err.message }
self._errors = {
'all': err.message
}
return None
return instance

13
orchestra/apps/domains/tests/functional_tests/tests.py
View File

@ -8,7 +8,7 @@ from django.core.urlresolvers import reverse
from selenium.webdriver.support.select import Select
from orchestra.apps.orchestration.models import Server, Route
from orchestra.utils.tests import BaseLiveServerTestCase, random_ascii, snapshot_on_error
from orchestra.utils.tests import BaseLiveServerTestCase, random_ascii, snapshot_on_error, save_response_on_error
from orchestra.utils.system import run
from ... import settings, utils, backends
@ -194,13 +194,13 @@ class DomainTestMixin(object):
self.add(self.ns1_name, self.ns1_records)
self.add(self.ns2_name, self.ns2_records)
self.add(self.domain_name, self.domain_records)
self.addCleanup(partial(self.delete, self.domain_name))
# self.addCleanup(partial(self.delete, self.domain_name))
self.update(self.domain_name, self.domain_update_records)
self.add(self.www_name, self.www_records)
# self.add(self.www_name, self.www_records)
time.sleep(0.5)
self.validate_update(self.MASTER_SERVER_ADDR, self.domain_name)
time.sleep(5)
self.validate_update(self.SLAVE_SERVER_ADDR, self.domain_name)
# time.sleep(5)
# self.validate_update(self.SLAVE_SERVER_ADDR, self.domain_name)
def test_add_add_delete_delete(self):
self.add(self.ns1_name, self.ns1_records)
@ -276,15 +276,18 @@ class RESTDomainMixin(DomainTestMixin):
self.rest_login()
self.add_route()
@save_response_on_error
def add(self, domain_name, records):
records = [ dict(type=type, value=value) for type,value in records ]
self.rest.domains.create(name=domain_name, records=records)
@save_response_on_error
def delete(self, domain_name):
domain = Domain.objects.get(name=domain_name)
domain = self.rest.domains.retrieve(id=domain.pk)
domain.delete()
@save_response_on_error
def update(self, domain_name, records):
records = [ dict(type=type, value=value) for type,value in records ]
domains = self.rest.domains.retrieve(name=domain_name)

6
orchestra/apps/lists/tests/functional_tests/tests.py
View File

@ -64,7 +64,7 @@ class ListMixin(object):
backend = backends.MailmanBackend.get_name()
Route.objects.create(backend=backend, match=True, host=server)
def atest_add(self):
def test_add(self):
name = '%s_list' % random_ascii(10)
password = '@!?%spppP001' % random_ascii(5)
admin_email = 'root@test3.orchestra.lan'
@ -100,8 +100,8 @@ class RESTListMixin(ListMixin):
self.rest.lists.create(name=name, password=password, admin_email=admin_email, **extra)
@save_response_on_error
def delete(self, username):
list = self.rest.lists.retrieve(name=username).get()
def delete(self, name):
list = self.rest.lists.retrieve(name=name).get()
list.delete()

1
orchestra/apps/orchestration/manager.py
View File

@ -14,6 +14,7 @@ logger = logging.getLogger(__name__)
def as_task(execute):
def wrapper(*args, **kwargs):
""" failures on the backend execution doesn't fuck the request transaction atomicity """
db.transaction.set_autocommit(False)
try:
log = execute(*args, **kwargs)

8
orchestra/apps/orchestration/middlewares.py
View File

@ -49,6 +49,7 @@ class OperationsMiddleware(object):
request = getattr(cls.thread_locals, 'request', None)
if request is None:
return
good_action = action
pending_operations = cls.get_pending_operations()
for backend in ServiceBackend.get_backends():
instance = None
@ -75,7 +76,7 @@ class OperationsMiddleware(object):
if update_fields:
# "update_fileds=[]" is a convention for explicitly executing backend
# i.e. account.disable()
if not update_fields == []:
if update_fields != []:
execute = False
for field in update_fields:
if field not in backend.ignore_fields:
@ -84,12 +85,17 @@ class OperationsMiddleware(object):
if not execute:
continue
instance = copy.copy(instance)
good = instance
operation = Operation.create(backend, instance, action)
if action != Operation.DELETE:
# usually we expect to be using last object state,
# except when we are deleting it
pending_operations.discard(operation)
pending_operations.add(operation)
try:
print kwargs['instance'], good_action
except:
pass
def process_request(self, request):
""" Store request on a thread local variable """

6
orchestra/apps/systemusers/tests/functional_tests/tests.py
View File

@ -82,9 +82,9 @@ class SystemUserMixin(object):
# Home will be deleted on account delete, see test_delete_account
def validate_ftp(self, username, password):
connection = ftplib.FTP(self.MASTER_SERVER)
connection.login(user=username, passwd=password)
connection.close()
ftp = ftplib.FTP(self.MASTER_SERVER)
ftp.login(user=username, passwd=password)
ftp.close()
def validate_sftp(self, username, password):
transport = paramiko.Transport((self.MASTER_SERVER, 22))

20
orchestra/apps/webapps/backends/__init__.py
View File

@ -1,23 +1,31 @@
import pkgutil
import textwrap
class WebAppServiceMixin(object):
model = 'webapps.WebApp'
def create_webapp_dir(self, context):
self.append("mkdir -p '%(app_path)s'" % context)
self.append("chown %(user)s.%(group)s '%(app_path)s'" % context)
self.append(textwrap.dedent("""
path=""
for dir in $(echo %(app_path)s | tr "/" "\n"); do
path="${path}/${dir}"
[ -d $path ] || {
mkdir "${path}"
chown %(user)s.%(group)s "${path}"
}
done
""" % context))
def delete_webapp_dir(self, context):
self.append("rm -fr %(app_path)s" % context)
def get_context(self, webapp):
return {
'user': webapp.account.user.username,
'group': webapp.account.user.username,
'user': webapp.account.username,
'group': webapp.account.username,
'app_name': webapp.name,
'type': webapp.type,
'app_path': webapp.get_path(),
'app_path': webapp.get_path().rstrip('/'),
'banner': self.get_banner(),
}

9
orchestra/apps/webapps/backends/awstats.py
View File

@ -1,9 +0,0 @@
from django.utils.translation import ugettext_lazy as _
from orchestra.apps.orchestration import ServiceController
from . import WebAppServiceMixin
class AwstatsBackend(WebAppServiceMixin, ServiceController):
verbose_name = _("Awstats")

26
orchestra/apps/webapps/backends/phpfcgid.py
View File

@ -1,4 +1,5 @@
import os
import textwrap
from django.utils.translation import ugettext_lazy as _
@ -15,9 +16,12 @@ class PHPFcgidBackend(WebAppServiceMixin, ServiceController):
context = self.get_context(webapp)
self.create_webapp_dir(context)
self.append("mkdir -p %(wrapper_dir)s" % context)
self.append(
"{ echo -e '%(wrapper_content)s' | diff -N -I'^\s*#' %(wrapper_path)s - ; } ||"
" { echo -e '%(wrapper_content)s' > %(wrapper_path)s; UPDATED=1; }" % context)
self.append(textwrap.dedent("""\
{
echo -e '%(wrapper_content)s' | diff -N -I'^\s*#' %(wrapper_path)s -
} || {
echo -e '%(wrapper_content)s' > %(wrapper_path)s; UPDATED_APACHE=1
}""" % context))
self.append("chmod +x %(wrapper_path)s" % context)
self.append("chown -R %(user)s.%(group)s %(wrapper_dir)s" % context)
@ -25,6 +29,10 @@ class PHPFcgidBackend(WebAppServiceMixin, ServiceController):
context = self.get_context(webapp)
self.delete_webapp_dir(context)
def commit(self):
super(PHPFcgidBackend, self).commit()
self.append("[[ $UPDATED_APACHE == 1 ]] && { /etc/init.d/apache reload; }")
def get_context(self, webapp):
context = super(PHPFcgidBackend, self).get_context(webapp)
init_vars = webapp.get_php_init_vars()
@ -36,12 +44,12 @@ class PHPFcgidBackend(WebAppServiceMixin, ServiceController):
context['init_vars'] = ''
wrapper_path = settings.WEBAPPS_FCGID_PATH % context
context.update({
'wrapper_content': (
"#!/bin/sh\n"
"# %(banner)s\n"
"export PHPRC=/etc/%(type)s/cgi/\n"
"exec /usr/bin/%(type)s-cgi %(init_vars)s\n"
) % context,
'wrapper_content': textwrap.dedent("""\
#!/bin/sh
# %(banner)s
export PHPRC=/etc/%(type)s/cgi/
exec /usr/bin/%(type)s-cgi %(init_vars)s
""" % context),
'wrapper_path': wrapper_path,
'wrapper_dir': os.path.dirname(wrapper_path),
})

2
orchestra/apps/webapps/backends/phpfpm.py
View File

@ -36,8 +36,8 @@ class PHPFPMBackend(WebAppServiceMixin, ServiceController):
})
context['fpm_listen'] = settings.WEBAPPS_FPM_LISTEN % context
fpm_config = Template(
"[{{ user }}]\n"
";; {{ banner }}\n"
"[{{ user }}]\n"
"user = {{ user }}\n"
"group = {{ group }}\n\n"
"listen = {{ fpm_listen | safe }}\n"

4
orchestra/apps/webapps/models.py
View File

@ -57,7 +57,7 @@ class WebApp(models.Model):
return init_vars
def get_fpm_port(self):
return settings.WEBAPPS_FPM_START_PORT + self.account.user.pk
return settings.WEBAPPS_FPM_START_PORT + self.account.pk
def get_method(self):
method = settings.WEBAPPS_TYPES[self.type]
@ -66,7 +66,7 @@ class WebApp(models.Model):
def get_path(self):
context = {
'user': self.account.user,
'user': self.account.username,
'app_name': self.name,
}
return settings.WEBAPPS_BASE_ROOT % context

0
orchestra/apps/webapps/tests/__init__.py Normal file
View File

0
orchestra/apps/webapps/tests/functional_tests/__init__.py Normal file
View File

181
orchestra/apps/webapps/tests/functional_tests/tests.py Normal file
View File

@ -0,0 +1,181 @@
import ftplib
import os
import time
import textwrap
from StringIO import StringIO
from django.conf import settings as djsettings
from django.contrib.contenttypes.models import ContentType
from django.core.management.base import CommandError
from django.core.urlresolvers import reverse
from selenium.webdriver.support.select import Select
from orchestra.apps.accounts.models import Account
from orchestra.apps.domains.models import Domain
from orchestra.apps.orchestration.models import Server, Route
from orchestra.apps.resources.models import Resource
from orchestra.apps.systemusers.backends import SystemUserBackend
from orchestra.utils.system import run, sshrun
from orchestra.utils.tests import BaseLiveServerTestCase, random_ascii, snapshot_on_error, save_response_on_error
from ... import backends, settings
from ...models import WebApp
class WebAppMixin(object):
MASTER_SERVER = os.environ.get('ORCHESTRA_MASTER_SERVER', 'localhost')
DEPENDENCIES = (
'orchestra.apps.orchestration',
'orchestra.apps.systemusers',
'orchestra.apps.webapps',
)
def setUp(self):
super(WebAppMixin, self).setUp()
self.add_route()
djsettings.DEBUG = True
def add_route(self):
# backends = [
# # TODO MU apps on SaaS?
# backends.awstats.AwstatsBackend,
# backends.dokuwikimu.DokuWikiMuBackend,
# backends.drupalmu.DrupalMuBackend,
# backends.phpfcgid.PHPFcgidBackend,
# backends.phpfpm.PHPFPMBackend,
# backends.static.StaticBackend,
# backends.wordpressmu.WordpressMuBackend,
# ]
server = Server.objects.create(name=self.MASTER_SERVER)
for backend in [SystemUserBackend, self.backend]:
backend = backend.get_name()
Route.objects.create(backend=backend, match=True, host=server)
def test_add(self):
name = '%s_%s_webapp' % (random_ascii(10), self.type_value)
self.add_webapp(name)
self.validate_add_webapp(name)
# self.addCleanup(self.delete, username)
class StaticWebAppMixin(object):
backend = backends.static.StaticBackend
type_value = 'static'
token = random_ascii(100)
page = (
'index.html',
'<html>Hello World! %s </html>\n' % token,
'<html>Hello World! %s </html>\n' % token,
)
def validate_add_webapp(self, name):
try:
ftp = ftplib.FTP(self.MASTER_SERVER)
ftp.login(user=self.account.username, passwd=self.account_password)
ftp.cwd('webapps/%s' % name)
index = StringIO()
index.write(self.page[1])
index.seek(0)
ftp.storbinary('STOR %s' % self.page[0], index)
index.close()
finally:
ftp.close()
class PHPFcidWebAppMixin(StaticWebAppMixin):
backend = backends.phpfcgid.PHPFcgidBackend
type_value = 'php5'
token = random_ascii(100)
page = (
'index.php',
'<?php print("Hello World! %s");\n?>\n' % token,
'Hello World! %s' % token,
)
class PHPFPMWebAppMixin(StaticWebAppMixin):
backend = backends.phpfpm.PHPFPMBackend
type_value = 'php5.5'
token = random_ascii(100)
page = (
'index.php',
'<?php print("Hello World! %s");\n?>\n' % token,
'Hello World! %s' % token,
)
class RESTWebAppMixin(object):
def setUp(self):
super(RESTWebAppMixin, self).setUp()
self.rest_login()
# create main user
self.save_systemuser()
@save_response_on_error
def save_systemuser(self):
self.rest.systemusers.retrieve().get().save()
@save_response_on_error
def add_webapp(self, name, options=[]):
self.rest.webapps.create(name=name, type=self.type_value)
@save_response_on_error
def delete_webapp(self, name):
list = self.rest.lists.retrieve(name=name).get()
list.delete()
class AdminWebAppMixin(WebAppMixin):
def setUp(self):
super(AdminWebAppMixin, self).setUp()
self.admin_login()
# create main user
self.save_systemuser()
# TODO save_account()
@snapshot_on_error
def add(self, name, password, admin_email):
url = self.live_server_url + reverse('admin:mails_List_add')
self.selenium.get(url)
account_input = self.selenium.find_element_by_id('id_account')
account_select = Select(account_input)
account_select.select_by_value(str(self.account.pk))
name_field = self.selenium.find_element_by_id('id_name')
name_field.send_keys(username)
password_field = self.selenium.find_element_by_id('id_password1')
password_field.send_keys(password)
password_field = self.selenium.find_element_by_id('id_password2')
password_field.send_keys(password)
if quota is not None:
quota_id = 'id_resources-resourcedata-content_type-object_id-0-allocated'
quota_field = self.selenium.find_element_by_id(quota_id)
quota_field.clear()
quota_field.send_keys(quota)
if filtering is not None:
filtering_input = self.selenium.find_element_by_id('id_filtering')
filtering_select = Select(filtering_input)
filtering_select.select_by_value("CUSTOM")
filtering_inline = self.selenium.find_element_by_id('fieldsetcollapser0')
filtering_inline.click()
time.sleep(0.5)
filtering_field = self.selenium.find_element_by_id('id_custom_filtering')
filtering_field.send_keys(filtering)
name_field.submit()
self.assertNotEqual(url, self.selenium.current_url)
class RESTWebAppTest(PHPFcidWebAppMixin, RESTWebAppMixin, WebAppMixin, BaseLiveServerTestCase):
pass
#class AdminWebAppTest(AdminWebAppMixin, BaseLiveServerTestCase):
# pass

64
orchestra/apps/websites/backends/apache.py
View File

@ -30,15 +30,13 @@ class Apache2Backend(ServiceController):
apache_conf = Template(textwrap.dedent("""\
# {{ banner }}
<VirtualHost *:{{ site.port }}>
ServerName {{ site.domains.all|first }}
ServerName {{ site.domains.all|first }}\
{% if site.domains.all|slice:"1:" %}
ServerAlias {{ site.domains.all|slice:"1:"|join:' ' }}
{% endif %}
ServerAlias {{ site.domains.all|slice:"1:"|join:' ' }}{% endif %}
CustomLog {{ logs }} common
SuexecUserGroup {{ user }} {{ group }}
{% for line in extra_conf.splitlines %}"
{{ line | safe }}
{% endfor %}
SuexecUserGroup {{ user }} {{ group }}\
{% for line in extra_conf.splitlines %}
{{ line | safe }}{% endfor %}
</VirtualHost>"""
))
apache_conf = apache_conf.render(Context(context))
@ -83,13 +81,13 @@ class Apache2Backend(ServiceController):
context = self.get_content_context(content)
context['fcgid_path'] = fcgid_path % context
fcgid = self.get_alias_directives(content)
fcgid += (
"ProxyPass %(location)s !\n"
"<Directory %(app_path)s>\n"
" Options +ExecCGI\n"
" AddHandler fcgid-script .php\n"
" FcgidWrapper %(fcgid_path)s\n"
) % context
fcgid += textwrap.dedent("""\
ProxyPass %(location)s !
<Directory %(app_path)s>
Options +ExecCGI
AddHandler fcgid-script .php
FcgidWrapper %(fcgid_path)s
""" % context)
for option in content.webapp.options.filter(name__startswith='Fcgid'):
fcgid += " %s %s\n" % (option.name, option.value)
fcgid += "</Directory>\n"
@ -100,11 +98,11 @@ class Apache2Backend(ServiceController):
custom_cert = site.options.filter(name='ssl')
if custom_cert:
cert = tuple(custom_cert[0].value.split())
directives = (
"SSLEngine on\n"
"SSLCertificateFile %s\n"
"SSLCertificateKeyFile %s\n"
) % cert
directives = textwrap.dedent("""\
SSLEngine on
SSLCertificateFile %s
SSLCertificateKeyFile %s""" % cert
)
return directives
def get_security(self, site):
@ -129,17 +127,17 @@ class Apache2Backend(ServiceController):
path, name, passwd = re.match(regex, protection.value).groups()
path = os.path.join(context['root'], path)
passwd = os.path.join(self.USER_HOME % context, passwd)
protections += ("\n"
"<Directory %s>\n"
" AllowOverride All\n"
# " AuthPAM_Enabled off\n"
" AuthType Basic\n"
" AuthName %s\n"
" AuthUserFile %s\n"
" <Limit GET POST>\n"
" require valid-user\n"
" </Limit>\n"
"</Directory>\n" % (path, name, passwd)
protections += textwrap.dedent("""
<Directory %s>
AllowOverride All
#AuthPAM_Enabled off
AuthType Basic
AuthName %s
AuthUserFile %s
<Limit GET POST>
require valid-user
</Limit>
</Directory>""" % (path, name, passwd)
)
return protections
@ -161,8 +159,8 @@ class Apache2Backend(ServiceController):
'site': site,
'site_name': site.name,
'site_unique_name': site.unique_name,
'user': site.account.user.username,
'group': site.account.user.username,
'user': site.account.username,
'group': site.account.username,
'sites_enabled': sites_enabled,
'sites_available': "%s.conf" % os.path.join(sites_available, site.unique_name),
'logs': os.path.join(settings.WEBSITES_BASE_APACHE_LOGS, site.unique_name),
@ -190,7 +188,7 @@ class Apache2Traffic(ServiceMonitor):
def prepare(self):
current_date = timezone.localtime(self.current_date)
current_date = current_date.strftime("%Y%m%d%H%M%S")
self.append(textwrap.dedent("""
self.append(textwrap.dedent("""\
function monitor () {
OBJECT_ID=$1
INI_DATE=$2

0
orchestra/apps/websites/tests/__init__.py Normal file
View File

0
orchestra/apps/websites/tests/functional_tests/__init__.py Normal file
View File

91
orchestra/apps/websites/tests/functional_tests/tests.py Normal file
View File

@ -0,0 +1,91 @@
import os
import socket
import time
import textwrap
from django.conf import settings as djsettings
from django.contrib.contenttypes.models import ContentType
from django.core.management.base import CommandError
from django.core.urlresolvers import reverse
import requests
from selenium.webdriver.support.select import Select
from orchestra.apps.accounts.models import Account
from orchestra.apps.domains.models import Domain, Record
from orchestra.apps.domains.backends import Bind9MasterDomainBackend
from orchestra.apps.orchestration.models import Server, Route
from orchestra.apps.resources.models import Resource
from orchestra.apps.webapps.tests.functional_tests.tests import StaticWebAppMixin, RESTWebAppMixin, WebAppMixin, PHPFcidWebAppMixin, PHPFPMWebAppMixin
from orchestra.utils.system import run, sshrun
from orchestra.utils.tests import BaseLiveServerTestCase, random_ascii, snapshot_on_error, save_response_on_error
from ... import backends, settings
from ...models import Website
class WebsiteMixin(WebAppMixin):
MASTER_SERVER = os.environ.get('ORCHESTRA_MASTER_SERVER', 'localhost')
MASTER_SERVER_ADDR = socket.gethostbyname(MASTER_SERVER)
DEPENDENCIES = (
'orchestra.apps.orchestration',
'orchestra.apps.domains',
'orchestra.apps.websites',
'orchestra.apps.webapps',
'orchestra.apps.systemusers',
)
def add_route(self):
super(WebsiteMixin, self).add_route()
server = Server.objects.get()
backend = backends.apache.Apache2Backend.get_name()
Route.objects.create(backend=backend, match=True, host=server)
backend = Bind9MasterDomainBackend.get_name()
Route.objects.create(backend=backend, match=True, host=server)
def validate_add_website(self, name, domain):
url = 'http://%s/%s' % (domain.name, self.page[0])
self.assertEqual(self.page[2], requests.get(url).content)
def test_add(self):
# TODO domains with "_" bad name!
domain_name = '%sdomain.lan' % random_ascii(10)
domain = Domain.objects.create(name=domain_name, account=self.account)
domain.records.create(type=Record.A, value=self.MASTER_SERVER_ADDR)
self.save_domain(domain)
webapp = '%s_%s_webapp' % (random_ascii(10), self.type_value)
self.add_webapp(webapp)
self.validate_add_webapp(webapp)
website = '%s_website' % random_ascii(10)
self.add_website(website, domain, webapp)
self.validate_add_website(website, domain)
class RESTWebsiteMixin(RESTWebAppMixin):
@save_response_on_error
def save_domain(self, domain):
self.rest.domains.retrieve().get().save()
def add_website(self, name, domain, webapp):
domain = self.rest.domains.retrieve().get()
webapp = self.rest.webapps.retrieve().get()
self.rest.websites.create(name=name, domains=[domain.url], contents=[{'webapp': webapp.url}])
#class RESTWebsiteTest(RESTWebsiteMixin, StaticWebAppMixin, WebsiteMixin, BaseLiveServerTestCase):
# pass
PHPFPMWebAppMixin
#class RESTWebsiteTest(RESTWebsiteMixin, PHPFcidWebAppMixin, WebsiteMixin, BaseLiveServerTestCase):
# pass
class RESTWebsiteTest(RESTWebsiteMixin, PHPFPMWebAppMixin, WebsiteMixin, BaseLiveServerTestCase):
pass
#class AdminWebsiteTest(AdminWebsiteMixin, BaseLiveServerTestCase):
# pass

2
orchestra/core/validators.py
View File

@ -41,7 +41,7 @@ def validate_name(value):
"""
A single non-empty line of free-form text with no whitespace.
"""
validators.RegexValidator('^\w+$',
validators.RegexValidator('^[\.\w]+$',
_("Enter a valid name (text without whitspaces)."), 'invalid')(value)

18
scripts/services/apache_full_stack.md
View File

@ -28,12 +28,14 @@ The goal of this setup is having a high-performance state-of-the-art deployment
apt-get install apache2-mpm-event php5-fpm libapache2-mod-fcgid apache2-suexec-custom php5-cgi
```
# TODO libapache2-mod-auth-pam is no longer part of the debian distribution,
# replace with libapache2-mod-authnz-external pwauth
2. Enable some convinient Apache modules
```bash
a2enmod suexec
a2enmod ssl
a2enmod auth_pam
#a2enmod auth_pam
a2enmod proxy_fcgi
a2emmod userdir
```
@ -55,15 +57,18 @@ The goal of this setup is having a high-performance state-of-the-art deployment
```
5. Restart Apache
```bash
service apache2 restart
```
* TODO
libapache2-mod-auth-pam
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=710770
* ExecCGI
@ -73,6 +78,11 @@ The goal of this setup is having a high-performance state-of-the-art deployment
</Directory>
```
* Permissions
<Directory /home/*/webapps>
Require all granted
</Directory>
TODO CHRoot
https://andrewbevitt.com/tutorials/apache-varnish-chrooted-php-fpm-wordpress-virtual-host/

10
scripts/services/vsftpd.md
View File

@ -10,10 +10,12 @@ VsFTPd with System Users
2. Make some configurations
```bash
sed -i "s/anonymous_enable=YES/anonymous_enable=NO/" /etc/vsftpd.conf
sed -i "s/#local_enable=YES/local_enable=YES/" /etc/vsftpd.conf
sed -i "s/#write_enable=YES/write_enable=YES/" /etc/vsftpd.conf
# sed -i "s/#chroot_local_user=YES/chroot_local_user=YES/" /etc/vsftpd.conf
sed -i "s/^anonymous_enable=YES/anonymous_enable=NO/" /etc/vsftpd.conf
sed -i "s/^#local_enable=YES/local_enable=YES/" /etc/vsftpd.conf
sed -i "s/^#write_enable=YES/write_enable=YES/" /etc/vsftpd.conf
# sed -i "s/^#chroot_local_user=YES/chroot_local_user=YES/" /etc/vsftpd.conf
sed -i "s/^#local_umask=022/local_umask=022/" /etc/vsftpd.conf
echo '/dev/null' >> /etc/shells
```