django-orchestra-test/TODO.md

328 lines
12 KiB
Markdown
Raw Normal View History

2014-11-18 13:59:21 +00:00
==== TODO ====
2014-05-08 16:59:35 +00:00
* scape strings before executing scripts in order to prevent exploits: django templates automatically scapes things. Most important is to ensuer that all escape ' to &quot
* Don't store passwords and other service parameters that can be changed by the services i.e. mailman, vps etc. Find an execution mechanism that trigger `change_password()`
* abort transaction on orchestration when `state == TIMEOUT` ?
* use format_html_join for orchestration email alerts
* enforce an emergency email contact and account to contact contacts about problems when mailserver is down
* add `BackendLog` retry action
* webmail identities and addresses
* Permissions .filter_queryset()
* env vars instead of multiple settings files: https://devcenter.heroku.com/articles/config-vars ?
2014-05-27 15:55:09 +00:00
* Log changes from rest api (serialized objects)
2014-07-11 14:48:46 +00:00
2014-07-18 15:32:27 +00:00
2014-07-24 09:53:34 +00:00
* backend logs with hal logo
2014-08-29 16:13:34 +00:00
* LAST version of this shit http://wkhtmltopdf.org/downloads.h otml
* translations
2015-03-04 21:06:16 +00:00
from django.utils import translation
with translation.override('en'):
2014-09-06 10:56:30 +00:00
* help_text on readonly_fields specialy Bill.state. (eg. A bill is in OPEN state when bla bla )
2014-09-19 14:47:25 +00:00
* create log file at /var/log/orchestra.log and rotate
2014-09-22 15:59:53 +00:00
* order.register_at
@property
def register_on(self):
return order.register_at.date()
2014-09-23 16:23:36 +00:00
2014-09-26 15:05:20 +00:00
* mail backend related_models = ('resources__content_type') ??
2014-09-26 19:21:09 +00:00
* Maildir billing tests/ webdisk billing tests (avg metric)
2014-09-28 12:28:57 +00:00
2015-03-11 20:01:08 +00:00
* when using modeladmin to store shit like self.account, make sure to have a cleanslate in each request? no, better reuse the last one
2014-09-28 12:28:57 +00:00
2015-03-11 20:01:08 +00:00
* jabber with mailbox accounts (dovecot mail notification)
2014-09-28 12:28:57 +00:00
2014-11-16 18:39:31 +00:00
* rename accounts register to "account", and reated api and admin references
2014-09-28 12:28:57 +00:00
2014-09-29 12:22:45 +00:00
* prevent deletion of main user by the user itself
2015-03-11 20:01:08 +00:00
* AccountAdminMixin auto adds 'account__name' on searchfields
2014-09-30 16:39:47 +00:00
* Separate panel from server passwords? Store passwords on panel? set_password special backend operation?
* What fields we really need on contacts? name email phone and what more?
2014-09-30 14:46:29 +00:00
* Redirect junk emails and delete every 30 days?
2014-10-03 14:02:11 +00:00
* DOC: Complitely decouples scripts execution, billing, service definition
* delete main user -> delete account or prevent delete main user
2014-10-03 14:02:11 +00:00
2015-03-11 20:01:08 +00:00
2014-10-09 17:04:12 +00:00
* multiple domains creation; line separated domains
2014-10-10 14:39:46 +00:00
2015-03-11 20:01:08 +00:00
* init.d celery scripts
-# Required-Start: $network $local_fs $remote_fs postgresql celeryd
-# Required-Stop: $network $local_fs $remote_fs postgresql celeryd
2015-03-11 20:01:08 +00:00
2014-10-15 21:18:50 +00:00
* regenerate virtual_domains every time (configure a separate file for orchestra on postfix)
2014-10-15 12:47:28 +00:00
* update_fields=[] doesn't trigger post save!
2014-10-15 21:18:50 +00:00
2014-10-17 10:04:47 +00:00
* Backend optimization
* fields = ()
* ignore_fields = ()
* based on a merge set of save(update_fields)
2014-10-17 20:03:41 +00:00
2014-10-18 12:26:59 +00:00
* parmiko write to a channel instead of transfering files? http://sysadmin.circularvale.com/programming/paramiko-channel-hangs/
2014-10-20 19:22:18 +00:00
2014-10-23 15:38:46 +00:00
* proforma without billing contact?
2015-03-11 20:01:08 +00:00
* print open invoices as proforma?
2015-03-29 16:10:07 +00:00
* env ORCHESTRA_MASTER_SERVER='test1.orchestra.lan' ORCHESTRA_SECOND_SERVER='test2.orchestra.lan' ORCHESTRA_SLAVE_SERVER='test3.orchestra.lan' python manage.py test orchestra.apps.domains.tests.functional_tests.tests:AdminBind9BackendDomainTest --nologcapture¶
2014-10-23 15:38:46 +00:00
* ForeignKey.swappable
* Field.editable
* ManyToManyField.symmetrical = False (user group)
* REST PERMISSIONS
2014-10-24 10:16:46 +00:00
2014-11-13 16:40:42 +00:00
* caching based on "def text2int(textnum, numwords={}):"
2014-10-27 13:29:02 +00:00
* multiple files monitoring
2014-10-30 16:34:02 +00:00
2014-11-02 14:33:55 +00:00
* sync() ServiceController method that synchronizes orchestra and servers (delete or import)
2014-11-13 16:40:42 +00:00
* consider removing mailbox support on forward (user@pangea.org instead)
2014-11-05 21:29:14 +00:00
2014-11-02 14:33:55 +00:00
* Databases.User add reverse M2M databases widget (like mailbox.addresses)
2014-11-05 21:29:14 +00:00
* reconsider binding webapps to systemusers (pangea multiple users wordpress-ftp, moodle-pangea, etc)
2014-11-05 20:22:01 +00:00
* Secondary user home in /home/secondaryuser and simlink to /home/main/webapps/app so it can have private storage?
2014-11-12 16:33:40 +00:00
* Grant permissions to systemusers, the problem of creating a related permission model is out of sync with the server-side. evaluate tradeoff
2014-11-05 20:22:01 +00:00
* Make one dedicated CGI user for each account only for CGI execution (fpm/fcgid). Different from the files owner, and without W permissions, so attackers can not inject backdors and malware.
* In most cases we can prevent the creation of files for the CGI users, preventing attackers to upload and executing PHPShells.
* Make main systemuser able to write/read everything on its home, including stuff created by the CGI user and secondary users
* Prevent users from accessing other users home while at the same time allow access Apache/fcgid/fpm and secondary users (x)
* resource min max allocation with validation
* mailman needs both aliases when address_name is provided (default messages and bounces and all)
2014-11-09 10:16:07 +00:00
* domain validation parse named-checzone output to assign errors to fields
* Directory Protection on webapp and use webapp path as base path (validate)
* User [Group] webapp/website option (validation) which overrides default mainsystemuser
2014-11-18 13:59:21 +00:00
* validate systemuser.home on server-side
* webapp backend option compatibility check?
2014-11-16 18:39:31 +00:00
* admin systemuser home/directory, add default home and empty directory with has_shell on admin
2014-11-18 13:59:21 +00:00
* Resource used_list_display=True, allocated_list_displat=True, allow resources to show up on list_display
* BackendLog.updated_at (tasks that run over several minutes when finished they do not appear first on the changelist) (like celery tasks.when)
2014-11-20 15:34:59 +00:00
* Periodic task for cleaning old monitoring data
* Create an admin service_view with icons (like SaaS app)
* Resource graph for each related object
2014-11-21 17:18:59 +00:00
* SaaS model splitted into SaaSUser and SaaSSite? inherit from SaaS
2014-11-27 19:17:26 +00:00
2015-02-24 09:34:26 +00:00
* prevent @pangea.org email addresses on contacts, enforce at least one email without @pangea.org
2014-12-22 11:40:02 +00:00
2015-03-02 10:37:25 +00:00
* forms autocomplete="off", doesn't work in chrome
2015-03-01 11:56:54 +00:00
ln -s /proc/self/fd /dev/fd
* escape passwords and not allow ' on them !
2015-03-01 11:56:54 +00:00
POST INSTALL
------------
* Generate a password-less ssh key, and copy it to the servers you want to orchestrate.
ssh-keygen
ssh-copy-id root@<server-address>
Php binaries should have this format: /usr/bin/php5.2-cgi
2015-03-04 21:06:16 +00:00
* logs on panel/logs/ ? mkdir ~webapps, backend post save signal?
* transaction fault tolerant on backend.execute()
* <IfModule security2_module> and other IfModule on backend SecRule
* Orchestra global search box on the page head, based https://github.com/django/django/blob/master/django/contrib/admin/options.py#L866 and iterating over all registered services and inspectin its admin.search_fields
2015-03-11 20:01:08 +00:00
2015-03-25 17:04:44 +00:00
* contain error on plugin missing key (plugin dissabled): NOP, fail hard is better than silently, perhaps fail at starttime? apploading machinary
2015-03-12 14:05:23 +00:00
* contact.alternative_phone on a phone.tooltip, email:to
* better validate options and directives (url locations, filesystem paths, etc..)
* make sure that you understand the risks
2015-03-18 21:51:12 +00:00
* full support for deactivation of services/accounts
2015-03-25 17:04:44 +00:00
* Display admin.is_active (disabled account special icon and order by support)
2015-03-18 21:51:12 +00:00
* lock resource monitoring
* -EXecCGI in common CMS upload locations /wp-upload/upload/uploads
* cgi user / pervent shell access
* prevent stderr when users exists on backend i.e. mysql user create
* disable anonymized list options (mailman)
2015-03-23 15:36:51 +00:00
* tags = GenericRelation(TaggedItem, related_query_name='bookmarks')
* make home for all systemusers (/home/username) and fix monitors
* user provided crons
* ```<?php
$moodle_host = $SERVER[HTTP_HOST];
require_once(/etc/moodles/.$moodle_host.config.php);``` moodle/drupla/php-list multi-tenancy
* make account available on all admin forms
* WPMU blog traffic
* normurlpath '' return '/'
2015-03-23 15:36:51 +00:00
2015-03-27 19:50:54 +00:00
* more robust backend error handling, continue executing but exit code > 0 if failure: failing_cmd || exit_code=1 and don't forget to call super.commit()!!
2015-03-23 15:36:51 +00:00
2015-03-27 19:50:54 +00:00
* website directives uniquenes validation on serializers
2015-03-27 19:50:54 +00:00
+ is_Active custom filter with support for instance.account.is_Active annotate with F() needed (django 1.8)
2015-03-27 19:50:54 +00:00
* delete apache logs and php logs
2015-03-27 19:50:54 +00:00
* document service help things: discount/refound/compensation effect and metric table
* Document metric interpretation help_text
* document plugin serialization, data_serializer?
* bill line managemente, remove, undo (only when possible), move, copy, paste
* budgets: no undo feature
* Autocomplete admin fields like <site_name>.phplist... with js
* autoexpand mailbox.filter according to filtering options
* allow empty metric pack for default rates? changes on rating algo
* IMPORTANT make sure no order is created for mailboxes that include disk? or just don't produce lines with cost == 0 or quantity 0 ? maybe minimal quantity for billing? like 0.1 ? or minimal price? per line or per bill?
2015-03-27 19:50:54 +00:00
* Improve performance of admin change lists with debug toolbar and prefech_related
* and miscellaneous.service.name == 'domini-registre'
* DOMINI REGISTRE MIGRATION SCRIPTS
2015-03-29 16:10:07 +00:00
* lines too long on invoice, double lines or cut, and make margin wider
* PHP_TIMEOUT env variable in sync with fcgid idle timeout
http://foaa.de/old-blog/2010/11/php-apache-and-fastcgi-a-comprehensive-overview/trackback/index.html#pni-top0
2015-03-29 16:10:07 +00:00
* payment methods icons
* use server.name | server.address on python backends, like gitlab instead of settings?
* saas change password feature (the only way of re.running a backend)
* TODO raise404, here and everywhere
* display subline links on billlines, to show that they exists.
* update service orders on a celery task? because it take alot
2015-03-29 16:10:07 +00:00
* billline quantity eval('10x100') instead of miningless description '(10*100)'
2015-04-03 10:14:45 +00:00
# FIXME do more test, make sure billed until doesn't get uodated whhen services are billed with les metric, and don't upgrade billed_until when undoing under this circumstances
* line 513: change threshold and one time service metric change should update last value if not billed, only record for recurring invoicing. postpay services should store the last metric for pricing period.
* add ini, end dates on bill lines and breakup quanity into size(defaut:1) and metric
* threshold for significative metric accountancy on services.handler
2015-04-01 15:49:21 +00:00
* http://orchestra.pangea.org/admin/orders/order/6418/
* http://orchestra.pangea.org/admin/orders/order/6495/bill_selected_orders/
2015-03-29 16:10:07 +00:00
* move normurlpath to orchestra.utils from websites.utils
* write down insights
* use english on services defs and so on, an translate them on render time
* websites directives get_location() and use it on last change view validation stage to compare with contents.location and also on the backend ?
* modeladmin Default filter + search isn't working, prepend filter when searching
* IMPORTANT do all modles.py TODOs and create migrations for finished apps
* create service templates based on urlqwargs with the most basic services.
* Base price: domini propi (all domains) + extra for other domains
2015-03-29 16:10:07 +00:00
2015-04-01 15:49:21 +00:00
* prepend ORCHESTRA_ to orchestra/settings.py
* rename backends with generic names to concrete services.. eg VsFTPdTraffic, UNIXSystemUser
2015-03-29 16:10:07 +00:00
Translation
-----------
mkdir locale
django-admin.py makemessages -l ca
django-admin.py compilemessages -l ca
https://docs.djangoproject.com/en/1.7/topics/i18n/translation/#joining-strings-string-concat
from django.utils.translation import ugettext
from django.utils import translation
translation.activate('ca')
ugettext("Description")
* saas validate_creation generic approach, for all backends. standard output
2015-03-29 16:10:07 +00:00
* html code x: &times;
2015-04-01 15:49:21 +00:00
2015-04-02 16:14:55 +00:00
* periodic task to cleanup backendlogs, monitor data and metricstorage
2015-04-01 15:49:21 +00:00
* create orchestrate databases.Database pk=1 -n --dry-run | --noinput --action save (default)|delete --backend name (limit to this backend) --help
* uwsgi --max-requests=5000 \ # respawn processes after serving 5000 requests and
celery max-tasks-per-child
* generate settings.py more like django (installed_apps, middlewares, etc,,,)
* postupgradeorchestra send signals in order to hook custom stuff
* make base home for systemusers that ara homed into main account systemuser
2015-04-02 16:14:55 +00:00
* user force_text instead of unicode for _()
* autoscale celery workers http://docs.celeryproject.org/en/latest/userguide/workers.html#autoscaling
* Delete transaction middleware
* webapp has_website list filter
glic3rinu's django-fluent-dashboard
2015-04-03 10:14:45 +00:00
* gevent is not ported to python3 :'(
* uwsgi python3
2015-04-03 13:03:08 +00:00
https://github.com/django-nose/django-nose/archive/master.zip
django_debug_toolbar-1.3.0-py2.py3-none-any.whl
2015-04-03 10:14:45 +00:00
2015-04-03 13:03:08 +00:00
# FIXME account deletion generates a integrity error
2015-04-03 10:14:45 +00:00
# FIXME what to do when deleting accounts? set fk null and fill a username charfield? issues, invoices.. we whant all this to go away?
* implement delete All related services
2015-04-03 13:03:08 +00:00
* address name change does not remove old one :P